Jiebo Luo's Home Page

Professor Jiebo Luo
Albert Arendt Hopeman Professor

Department of Computer Science
3101 Wegmans Hall
University of Rochester
Rochester, NY 14627

Email: username -at- domain.edu
where username = jluo,
domain = cs.rochester.edu

Phone: 585-276-5784
Prospective Students

Computer Vision Urban Computing
Health/Wellness Analytics
Image Retrieval Image Segmentation
Scene Classification Big Data
Contextual Inference Machine
Learning Vision+Language
Random Field Photo Collection
Multimedia Video Analytics
Human-Centered Media & Interaction
Mobile Media Location-based service
Geo Tagging Action Recognition
Heterogeneous Information Networks
Social Multimedia Data Mining
Crowdsourcing 360D User Profiling

News

2024

11/14/2024: Prof. Luo is selected as a recipient of the 2025 Edward J. McCluskey Technical Achievement Award by the IEEE Computer Society
11/04/2024: Prof. Luo is elected to Fellow of AIMBE (American Institute for Medical and Biological Engineering)
10/30/2024: Prof. Luo is a Keynote Speaker at the 2024 ACM Multimedia Conference
10/16/2024: Prof. Luo wins the inaugural Debra Haring Excellence in Research Award, which recognizes "a faculty member who has done research that has had a transformational impact on the field, or who has accumulated a body of research that has changed the field in positive ways."
10/10/2024: Some members of VIStA watched the magnificent aurora together off Webster Pier in Rochester, NY

09/12/2024: Prof. Luo is an invited speaker at the Global AI Summit (GAIN)
08/29/2024: Congratulations to Hanjia Lyu for being awarded the 2024 Google PhD Fellowship

08/27/2024: IEEE CS releases an interview with Prof. Luo at CVPR 2024 on industry vs. academic careers, why publishing, why attending conferences, how to choose research topics, and so on.
08/07/2024: Our study that shows the tech community is split over AI is covered by Futurity, Quantum Zeitgeist, and CODU Compute.
05/18/2024: Our study on the factors contributing to crowdfunding success is featured in the Wall Street Journal
05/15/2024: Prof. Luo receives another major award - the 2024 Edmund A. Hajim Outstanding Faculty Award
04/08/2024: (Day of Total Solar Eclipse): Prof. Luo receives the 2024 William H. Riker University Award for Excellence in Graduate Teaching, which recognizes a faculty member who has excelled in graduate instruction, particularly in the University's doctoral programs.

03/28/2024: Congratulations to Alexander Martin for winning the 2024 Charles L. Newton Prize (awarded to senior engineering students who show a special proficiency in an engineering subject and have conducted research, given a presentation, or published a paper)
01/15/2024: Congratulations to Alexander Martin for an Honorable Mention in the 2023-2024 Computing Research Association's Outstanding Undergraduate Researcher Award Competition
01/04/2024: We release a joint survey of Video Understanding with Large Language Models

2023

12/12/2023: We release a joint survey of Large Language Models in Medicine: Principles, Applications, and Challenges

11/15/2023: We release an evaluation report on GPT-4V(ision) as A Social Media Analysis Engine

10/25/2023: Professor Luo serves as General Co-Chair of IEEE ICME 2024 (Call for Papers) in Niagara Falls

09/18/2023: News 10 NBC WHEC-TV Rochester features our NIH and NSF funded project SMARTeeth in their In-Depth Investigative program entitled "Bringing the dentist to you: UR developing app to identify tooth decay in children"; also reported by News 8 WROC-TV in their Rochester First program entitled "Eastman Institute using AI creates app for early tooth decay detection"

09/13/2023: We receive the Adobe 2023 Data Science Research Award to support our data science research

07/13/2023: In our new studies (ICWSM 2024, ICWSMW 2023), we uncover media bias in 1.8 million headlines across the political spectrum. Our findings suggest that media bias is growing. The work is covered by StudyFinds, and shared by ScienceBlog, RealClearScience, Nation, Mirage.News, Phys.org, and the American Association for the Advancement of Science (AAAS) via EurekAlert!

07/10/2023: Congratulations to Patrick Jingyuan Chen (a rising junior) for winning the Best Student Paper Award at the 2023 IEEE International Conference on Digital Health (ICDH) for the work on "sDREAMER: Self-distilled Hierarchical Mixture of Modality Experts Transformer are Efficient Mice Sleep Learners"
06/29/2023: Congrats to newly minted Dr. Songyang Zhang who is joining Amazon Research

06/17/2023: Professor Luo receives the 2023 Contribution Award from Health Data Science (a Science partner journal) "in recognition of contributions to Health Data Science"
05/22/2023: Congrats to newly minted Dr. Wei Zhu who has joined Amazon Research
05/19/2023: Congrats to newly minted Dr. Haitian Zheng who is joining Adobe Research

05/12/2023: Congratulations to Peirong Hao for receiving the CS Department Best Senior Award

05/12/2023: Congratulations to Yufei Zhao for receiving the CS Excellence in Undergraduate Research Award and Donald M. Barnard Prize (awarded to engineering students on the basis of personal qualification and achievement)

03/09/2023: Congratulations to Hanjia Lyu for being awarded the 2023 Arts, Sciences and Engineering Donald M. and Janet C. Barnard Fellowship

2022

12/08/2022: Professor Luo is elected to Fellow of NAI (National Academy of Inventors) for "have demonstrated a highly prolific spirit of innovation in creating or facilitating outstanding inventions that have made a tangible impact on the quality of life, economic development, and welfare of society"

10/10/2022: Congrats to Zhengyuan Yang, who graduated in 2021 from URCS and is now a Senior Researcher with Microsoft Research, for winning the 2022 ACM SIGMM Award for Outstanding PhD Thesis for his PhD thesis entitled "Visual Grounding: Building Cross-Modal Visual-Text Alignment"

09/01/2022: Professor Luo is recognized as the Albert Arendt Hopeman Professor of Engineering

08/01/2022: Professor Luo's group joins the Data Score of the NIH U19 project on Neural Circuit Control of Fluid and Solute Clearance During Sleep (by UR, MIT, and UPenn under the NIH Brain Initiative)

06/17/2022: Professor Luo is elected to a Member of Academia Europaea, a pan-European academy of humanities, law, and science based on sustained academic excellence in his field, as well as significant Europe-wide activities
06/14/2022: Congrats to newly minted Dr. Wei Xiong who is joining Adobe Photoshop
06/09/2022: Congrats to newly minted Dr. Weijian Li who is joining Amazon Go

03/25/2022: Our study on the scale and scope of the influence of misinformation and fact-based news about COVID-19 vaccines on social media platforms on the vaccine uptake is covered by multiple news outlets including Medical Economics, News Medical, News Wise, UK Today News, and Medical Xpress
02/01/2022: Application is open for NSF Summer REU on Imaging in Medicine and Biology for Underrepresented Minorities at University of Rochester

2021

10/22/2021: Professor Luo receives the 2021 ACM SIGMM Award for Outstanding Technical Contributions to Multimedia Computing, Communications and Applications for his "outstanding, pioneering and continued research contributions in the areas of multimedia content analysis and social media analytics and for outstanding and continued service to the multimedia community"

09/21/2021: Professor Luo reaches a milestone of 100 in his h-index according to Google Scholar

07/08/2021: Professor Luo delivers a keynote on "Vision and Language: The Past, Present and Future" at the 2021 IEEE International Conference on Multimedia and Expo (ICME). The slides are available in slideshare.net.
06/25/2021: Congrats to newly minted Dr. Tianlang Chen who has joined Amazon
06/07/2021: Congrats to newly minted Dr. Zhengyuan Yang who has joined Microsoft Research

05/26/2021: Congratulations to Songyang Zhang for winning the NAACL Best Long Paper Award at the 2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) for the work on "Video-aided Unsupervised Grammar Induction"

05/19/2021: In a new study, we dissect the public responses to the #StopAsianHate movement using large scale Twitter data and provide findings that can help design better ways of reducing tension and misunderstandings between ethnic groups. The work is also covered by Rochester Beacon and Futurity.
01/29/2021: U.S. Senate Majority Leader Charles E. Schumer and U.S. Senator Kirsten Gillibrand today announced federal funding to support the research and development of oral disease detection and intervention technology at the University of Rochester

2020

12/15/2020: Congratulations to Yipeng Zhang, who is selected for Honorable Mention for the Computing Research Association's (CRA) Outstanding Undergraduate Researcher Award for 2021

12/12/2020: The latest CS Rankings data show that in the area of Computer Vision, Rochester ranks 8th in the United States for the year of 2020
11/27/2020: Data Skeptic, a popular data science podcast reaching 30k+ listeners nationwide per week, airs a full episode on our recent work on the U.S. facemask sentiment analysis using Twitter data. The recording is available online and via Spotify.
07/23/2020: Tech Xplore reports a series of our recent studies that show Twitter mirrors our attitudes and feelings about COVID-19. These findings are shared in a lightning presentation for chairs of computer science PhD-granting departments and heads of major industrial research labs at the Computing Research Association (CRA) Virtual Conference.
05/30/2020: Our work on analyzing the use of controversial terms of COVID-19 on Twitter is highlighted by IEEE Spectrum, the flagship magazine and website of the IEEE. Incidentally, this story shares the headlines with the story about the successful launch of the SpaceX Dragon 2 spacecraft. Cool!
04/27/2020: Congrats to newly minted Dr. Yang Feng who will join Google Cloud

01/01/2020: Professor Luo begins his 3-year term as the Editor-in-Chief of the IEEE Transactions on Multimedia

2019

12/31/2019: The latest CS Rankings data show that in the area of Computer Vision, Rochester ranks in the Top 5 in the United States for the year of 2019, trailing only CMU, Stanford, UCFand Berkeley, and right ahead of MIT and Cornell

12/12/2019: Congratulations to Zhengyuan Yang for winning the 2020 Twitch Research Fellowship to support his PhD research on video and language. Twitch is the No. 1 live streaming video service for video games in the US, operated by Twitch Interactive, a subsidiary of Amazon. In addition, it offers music broadcasts, creative content, and more recently, "in real life" streams.
11/08/2019: Congrats to newly minted Dr. Haofu Liao who will join Amazon (AWS)
10/23/2019: Congrats to Songyang Zhang for winning the 1st place in the HACS Action Localization Challenge at ICCV 2019
10/17/2019: Our team wins the 1st place in the Challenge on Automatic Generation of Cardiovascular Diagnostic Report (AutoGen-CDR19) at MICCAI 2019
10/08/2019: Congratulations to Haofu Liao for being selected as a finalist for MICCAI 2019 Young Scientist Award based his work on Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction
09/13/2019: Congrats to newly minted Dr. Jianbo Yuan who will join Amazon (AWS)
08/08/2019: Congratulations to Weijian Li for being selected for a graduate student travel award on the basis of the score obtained during the review process. Weijian will present papers at MICCAI 2019. Yipeng Zhang also receives an undergraduate student travel award.
07/29/2019: Yipeng Zhang, a CS sophomore and a Xerox undergraduate engineering fellow in this summer, is featured in an Instagram post by our new University President Sarah C. Mangelsdorf. It includes a picture of Yipeng
with Sarah in front of Yipeng's poster on "A deep learning approach to early childhood caries prevention.
06/18/2019: Congratulations to Yang Feng who participates in the CVPR 2019 Doctoral Consortium
04/03/2019: Congrats to newly minted Dr. Tianran Hu who will join College of William and Mary as Assistant Professor of Computer Science

2018

12/07/2018: Professor Luo is elevated to Fellow of AAAI (Association for Advancement of Artificial Intelligence) for "significant contributions to the fields of computer vision and data mining, and particularly pioneering work on multimodal understanding for sentiment analysis, computational social science, and digital health"
11/01/2018: Professor Luo is elevated to Fellow of ACM (Association for Computing Machinery) for "contributions to multimedia content analysis and social multimedia informatics"

10/03/2018: The University of Rochester has been selected as a Morris K. Udall Center of Excellence in Parkinson's Disease Research by the National Institute of Neurological Disorders and Stroke (NINDS). The new $9.2 million award brings together researchers from industry and multiple academic institutions to focus on the development of digital tools to enhance understanding of the disease, engage broad populations in research, and accelerate the development of new treatments for Parkinson's disease. Professor Luo will co-lead the Advanced Analytics Core that brings together a team of experts in biostatistics, computational science and mathematics from multiple institutions.

09/15/2018: Professor Luo is the recipient of the 2018 IEEE Region 1 Technological Innovation in Academic Award for "contributions in computer vision and data mining"

08/24/2018: Congratulations to Zhengyuan Yang and Yixuan Zhang for winning Best Industrial Related Paper Award at ICPR 2018 for their paper on "End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception"
08/13/2018: Congrats to newly minted Dr. Yu Wang who has joined Apple
08/06/2018: Congratulations to Tianran Hu for being invited to the 2018 ACM Multimedia Doctoral Symposium
06/29/2018: Congratulations to Haofu Liao for being "pre-selected for a student travel award on the basis of the score your paper has obtained during the review process". Haofu will present two papers (both with early acceptance) at MICCAI 2018.

2017

06/22/2017: Professor Luo is an invited speaker at CVPR 2018 Workshop on Towards Automatic Understanding of Visual Advertisements
06/21/2017: Professor Luo hosts the 3rd annual "The Vision Seeker" (视觉求索) gathering at CVPR 2018

                    Scan QR Code via WeChat
                    to follow Official Account

02/13/2017: Professor Luo and multidisciplinary team are featured in a report by the University of Rochester Newscenter, "App combines virtual reality with personalized psychotherapy"
12/25/2017: Professor Luo is interviewed by Sixthtone in a feature "Even China's Backwater Cities Are Going Smart" which reveals China's grand plans for an artificial intelligence rollout of smart cities.
11/22/2017: Prof. Homer Chen visits our lab
10/13/2017: Congrats to newly minted Dr. Yuncheng Li who has joined Snapchat Research
08/29/2017: Congrats to newly minted Dr. Quanzeng You who is joining Microsoft Research AI

08/21/2017: Professor Luo is quoted by Market Mogul in a feature "Here's How Big Data Transformation Will Sweep Information Sovereignty"
08/14/2017: Professor Luo is interviewed by Bloomberg Technology in a feature"China's Plan for World Domination in AI Isn't So Crazy After All"
07/26/2017: Professor Luo is an invited speaker at CVPR 2017 Workshop on Medical Computer Vision and Health Informatics

07/21/2017: Professor Luo is co-organizing Language and Vision Workshop at CVPR 2017
07/21/2017: Professor Luo is co-teaching CVPR 2017 Tutorial (lecture notes online) on Dealing with Reality: Low-Quality Visual Data Processing and Analytics
05/29/2017: Professor Luo is interviewed by Xinhua News Agency on the heels of AlphaGo's triumph over Ke Jie Spotlight: Artificial intelligence fuels visions of how smart future can be
03/20/2017: Our work on Detecting the Hate Code on Social Media is covered extensively by Geek.com, Vice.com, Digital Trends, MSN, Thestack.com Our work is especially timely given that 1) YouTube is facing a full-scale advertising boycott over hate speech, and 2) Tech Giants Vow to Tackle Online Hate Speech Within 24 Hours

02/24/2017: Professor Luo's Data Mining course was considered "favorite class taken at the University of Rochester" in the Data Science program (Featured Article: Data science for a better planet)
02/11/2017: National Science Foundation Research Experience for Undergraduates (NSF REU) Computational Methods for Understanding Music, Media, and Minds. Please note that the projects are flexible and the current descriptions are meant as examples.

2016

11/07/2016: Professor Luo is a panelist on NPR Radio, WXXI Connections hosted by Evan Dawson Monthly Science Roundtable "Twitter, Social Media and The Election"
10/21/2016: Our work on fine-grained recognition of East Asian faces is covered by Washington Post. You are welcome to take the Asian Face Challenge at http://www.vista-today.com/asianmyth to help benchmark human recognition performance (currently at 49% with 400+ tests). Another story on NextShark

10/01/2016: Professor Luo speaks at the Huang Symposium @UIUC to celebrate the 80th birthday and extraordinary career of Professor Thomas S. Huang

08/29/2016: Congratulations to Tianran Hu who receives a student travel scholarship to attend CIKM 2016.
07/20/2016: URCS is highly ranked among the Top universities in Artificial Intelligence for the period of 2000-2016 by an objective measure of faculty publications
07/19/2016: URCS is among the Top 20 Artificial Intelligence Engineering Schools in the U.S.

07/01/2016: Congratulations to Quanzeng You for being invited to the 2016 ACM Multimedia Doctoral Symposium with a travel scholarship
06/26/2016: Yiheng Zhou and Numair Sani, our undergraduate researchers, present their work on mining Instagram for illicit drug use patterns at SBP-BRiMS 2016, supported by student travel scholarship
06/09/2016: Our work on "Will Sanders Supporters Jump Ship to Trump?" is covered by Pacific Standard
05/10/2016: Our work on using AI to hunt drug dealers on Instagram is covered by Daily Dot
05/04/2016: Our work on "Voting with Feet: Who are leaving Hilary Clinton and Donald Trump" is covered by AOL, Daily Dot, LipTV , Pivot America
04/21/2016: Zhishen Pan, a Junior student in the Data Science program, won the Poster Presentation Award at the 2nd Annual Rochester Global Health Symposium & UNYTE Scientific Session: Innovative Solutions to Combat Global Health Disparities for the poster "Towards Understanding How News Coverage Affects Public Perception During Epidemic Outbreak." Congratulations!
04/18/2016: Yu Wang receives a student travel scholarship to attend ICWSM-2016

03/29/2016: Congratulations to Yuncheng Li, who has been invited to the CVPR 2016 Doctoral Consortium
03/20/2016: Our work on using Twitter user avatar facial analysis to study Trump and Clinton followers is covered by MIT Technology Review, Digital Trends, Market Exclusive
03/17/2016: Our CVPR 2016 paper - "Image Captioning with Semantic Attention" - describes the engine behind a system that has been sitting atop the Leader Board for the Microsoft COCO (Common Objects in Context) Image Captioning Challenge since November 2015, beating Google, Microsoft, Baidu/UCLA, Stanford, UC Berkeley, University of Toronto/Montreal, and other power houses. Full Story
03/16/2016: Our work on detecting drinking-while-tweeting patterns is covered by MIT Technology Review, Gizmodo, CBS News, NPR Weekend Edition, Engadget, and more
02/06/2016: Our work on estimating city air pollution from crowd-sourced photos is covered by New Scientist Try our demo system

2015

12/21/2015: Quanzeng You is a Finalist for the inaugural Adobe Research Fellowship
11/01/2015: Our work on using Instagram posts to determine underage drinking patterns has been covered by Innovation Trail, Digital Trends, Democratic and Chronicle, TechRadar, Medical Daily, The Science Explorer, iTech Post
10/01/2015: Yu Wang and Tianran Hu receive student travel scholarships to attend IEEE ICDM 2015
10/02/2015: Data Science Faculty Search (2015-2016): neurally-inspired computing; computer vision for medicine and humanities
10/01/2015: Yu Wang receives a student travel scholarship to attend IEEE BigData 2015
08/28/2015: Our data-driven study on the influence of fashion shows on everyday trends has been featured by New York Post, Science News, El Pais (major Spanish Newspaper), MIT Technology Review

08/20/2015: Yuncheng Li et al. win the Best Paper Award at ICIMCS 2015 for the work on Using User Generated Photos to Monitor Air Pollution in Cities
07/20/2015: Deadline of the International Workshop on Social Multimedia Data Mining at ICDM 2015 (Atlantic City)

07/14/2015: Quanzeng You is invited to the 2015 Microsoft Research Asia Ph.D. Forum among "30 of the best Asian Ph.D. students from the Asia-Pacific region and the U.S."
05/15/2015: Prof. Luo is named a Data Science CoE Distinguished Researcher @Goergen Institute for Data Science
05/12/2015: Tianran Hu receives a student travel scholarship to attend ICWSM-2015
02/15/2015 Our work on computational image sentiment analysis is covered by Tech Times, Science 2.0
02/03/2015: Communications of ACM coverage of our Non-Obtrusive Mental Health Monitoring work; also Gizmag

2011-2014

12/15/2014: ICDM Tutorial on "Social Multimedia as Sensors"
12/14/2014: Workshop on Social Multimedia Data Mining at ICDM 2014
12/07/2014: Quanzeng You receives a student travel scholarship to attend AAAI-2015
10/02/2014: Prof. James Rehg visits our lab
08/28/2014: Guess how many students I currently advise?
07/15/2014: 2014 IEEE Multimedia Prize Paper Award

06/28/2014: The New Big Three of Rochester (Democratic and Chronicle interview)
05/01/2014: How Good is URCS? (a little known fact that puts us ahead of a perennial top 5 CS department)

12/20/2013: Prof. Roland Goecke visits our lab
10/25/2013: Prof. Bill Freeman visits our lab
10/18/2013: RoCData II (Rochester Big Data Forum)
10/17/2013: Prof. Ruslan Salakhutdinov visits our lab
10/14/2013: Prof. Honglak Lee visits our lab
08/29/2013: Prof. Eckehard Steinbach visits our lab
10/18/2013: RoCData II (Rochester Big Data Forum)
05/15/2013: The TMM Special Issue on "Social Media as Sensors" is complete and in press for August 2013

04/08/2013: A New Publication highlighting Big Data research at the UR
12/31/2012: URCS Faculty Recruiting in HCI and Big Data
10/04/2012: RoCData (Rochester Big Data Forum, speaker list)
09/19/2012: "Computer Vision with a Third Eye"

09/10/2012: VIStA (Visual Intelligence & Social Multimedia Analytics) Research Group is formed

02/25/2012: Professor Luo will co-chair the IEEE CVPR 2012 Area Chair meeting in Long Beach, CA

12/08/2011: Professor Luo will serve on the program committee for both the research track and industry track of the 2012 ACM SIGKDD conference
09/06/2011: I am looking for two self-motivated PhD students to join my research group in Fall 2012 to work in the areas of computer vision and social media data mining. Applicants are expected to have strong skills in both programming and mathematics. Related research publications in ranked venues of computer vision, machine learning, or data mining is a plus. To apply, check http://www.cs.rochester.edu/education/phd-program.

09/01/2011: Professor Luo joined the CS Department after a prolific career of 15+ years at Kodak Research Labs.
09/01/2011: Professor Luo will co-chair the Third ACM SIGMM Workshop on Social Media (WSM2011) in conjunction with ACM Multimedia 2011
09/01/2011: A feature article on "Aesthetics and Emotions in Images: A Computational Perspective" is published in the IEEE Signal Processing Magazine
09/01/2011: A mini book on Interactive Co-segmentation of Objects in Image Collections (SpringerBriefs in Computer Science) by Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, and Tsuhan Chen is available for pre-order
04/06/2011: A new book onSocial Media Modeling and Computingby Steven C.H. Hoi, Jiebo Luo, Susanne Boll and Dong Xu is published by Springer

Biography

Jiebo Luo joined the University of Rochester in Fall 2011 after over fifteen prolific years at Kodak Research Laboratories, where he was a Senior Principal Scientist leading research and advanced development. He has been involved in numerous technical conferences, including serving as general co-chair of ACM Multimedia 2018 and IEEE ICME 2024, and program co-chair of ACM Multimedia 2010, IEEE CVPR 2012 and IEEE ICIP 2017. He has served on the editorial boards of the IEEE Transactions on Pattern Analysis and Machine Intelligence, IEEE Transactions on Multimedia, IEEE Transactions on Circuits and Systems for Video Technology, ACM Transactions on Intelligent Systems and Technology, Pattern Recognition, Machine Vision and Applications, Intelligent Medicine, and Health Data Science. Dr. Luo served as the Editor-in-Chief of the IEEE Transactions on Multimedia for the 2020-2022 term. Dr. Luo is a Fellow of ACM, AAAI, IEEE, AIMBE, IAPR, and SPIE, as well as a Member of Academia Europaea (AE) and the US National Academy of Inventors (NAI). He was the winner of the ACM SIGMM Technical Achievement Award in 2021, the William H. Riker University Award for Excellence in Graduate Teaching in 2024, the Edmund A. Hajim Outstanding Faculty Award in 2024, and the inaugural Debra Haring Excellence in Research Award in 2024.

His research spans image processing, computer vision, machine learning, data mining, social media, computational social science, and digital health. He is a pioneer for contextual inference in semantic understanding of visual data and social multimedia data mining. He has published extensively in these fields with 600+ peer-reviewed technical papers and over 90+ US patents. His Google Scholar h-index is 127+ with over 60,000 citations. His research has been supported by federal and state agencies including NSF, NIH, ONR, and New York State CoE, as well as various corporate and foundation sponsors.

Teaching

Major Professional Services

IEEE:

IEEE Transactions on Multimedia (TMM), Editor-in-Chief (2020-2022), Editorial Board Member (2004-2008; 2013-2015)
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Editorial Board Member (2006-2011)

IEEE Transactions on Circuits and Systems for Video Technology (TCSVT), Editorial Board Member (2010-2012)
IEEE Transactions on Big Data (TBD), Editorial Board Member (2018-2022)

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Program Co-Chair (2012), Area Chair (2008/2017/2019)

IEEE International Conference on Image Processing (ICIP), Program Co-Chair (2017), Area Chair (2008-2012)
IEEE International Conference on Acoustics, Speech & Signal Processing (ICASSP), Area Chair (2009-2012)
IEEE International Conference on Multimedia and Expo (ICME), Special Sessions Chair (2006), General Co-Chair (2024)

IEEE Signal Processing Society Technical Committees: MLSP (2005-2010), MMSP (2006-2012), IVMSP (2007-2013)

IEEE International Conference on Data Mining (ICDM), Program Committee (2013-2022), Senior PC member (2015)

ACM

ACM Multimedia Conference, General Co-Chair (2018), Program Co-Chair (2010), Awards Chair (2024), Workshops Co-Chair (2012), Tutorial Co-Chair (2011)

ACM Transactions on Intelligent Systems and Technology, Editorial Board Member (2015-2024)

ACM/IAPR International Conference on Content-based Image and Video Retrieval (CIVR), General Chair (2008)

ACM International Conference on Multimedia Retrieval (ICMR), Program Co-Chair (2016), General Co-Chair (2023)

ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Senior PC member (2018-2020)
ACM Web Conference (formerly WWW), PC member (2020- )

ACM SIGMM Workshop on Social Media (WSM), Founding Chair (2009-2011)

ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Program Committee (2012- )

ACM Conference on Information and Knowledge Management (CIKM), Program Committee (2013- )

ACM International Conference on Multimedia Retrieval (ICMR), Steering Committee (2009- )

AAAI

International Conference on Artificial Intelligence (AAAI), Program Committee (2013-2015), Senior PC / AC (2016- )

International Joint Conference on Artificial Intelligence (IJCAI), Program Committee (2013-), Senior PC (2015/2018)

International Conference on Web and Social Media (ICWSM), Program Committee (2018), Senior PC (2019- )

SPIE

SPIE International Symposium on Visual Communication and Image Processing (VCIP), Co-Chair (2007)
SPIE International Symposium on Multimedia Systems & Applications, General Co-Chair (2007)

Journal of Electronic Imaging (JEI), Editorial Board Member (2003-2013)

CVF
- CVF/IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Program Co-Chair (2012), Area Chair (2008/2017/2019)
- International Conference on Computer Vision (ICCV), Area Chair (2011)
- European Conference on Computer Vision, Area Chair (2020)
IAPR

Pattern Recognition, Editorial Board Member (2002-2019)

International Conference on Pattern Recognition (ICPR), Area Chair (2014, 2016, 2018)

MICCAI

International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Area Chair (2020)

ACL

Annual meeting of the Association for Computational Linguistics (ACL), Program Committee (2020- )
Conference on Empirical Methods in Natural Language Processing (EMNLP), Program Committee (2020- )

NSF:

CISE-IIS-RI, panelist

CISE-IIS-III, panelist

SCH, panelist

VEC, panelist

CPS, panelist

Industry:

Board Member, Greater Rochester Data Science Industry Consortium (2018-2019)
Advisor, Tencent (Youtu) (2017)
Visiting Professor, Tencent AI Lab (2018)
Visiting Professor, Facebook AI (2021-2022)

Books

Deep Neural Network for Medical Image Computing: Principles and Applications (Elsevier, 2022)
Social Media Modeling and Computing (Springer)
Interactive Co-segmentation of Objects in Image Collections (SpringerBriefs in Computer Science)
Multimedia Interaction and Intelligent User Interfaces (Springer Advances in Computer Vision and Pattern Recognition)
Computer Vision [in English])[amazon.cn]

Selected Recent Publications (Publications listed more comprehensively by DBLP)

My Citations

surveys

Large Language Models in Medicine: Principles, Applications, and Challenges

Video Understanding with Large Language Models: A Survey

computer vision, NLP and media understanding and generation

2025

Hang Hua*, Yunlong Tang*, Chenliang Xu, Jiebo Luo, "V2Xum-LLM: Cross-Modal Video Summarization with Temporal Prompt Instruction Tuning," The 38th AAAI Conference on Artificial Intelligence (AAAI), Philadelphia, PA, February 2025.
Jingyuan Chen, Fuchen Long, Jie An, Zhaofan Qiu, Ting Yao, Jiebo Luo, Tao Mei, "Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion," The 38th AAAI Conference on Artificial Intelligence (AAAI), Philadelphia, PA, February 2025.
Hanjia Lyu, Jinfa Huang, Daoan Zhang, Yongsheng Yu, Xinyi Mou, Jinsheng Pan, Zhengyuan Yang, Zhongyu Wei, Jiebo Luo, "GPT-4V(ision) as A Social Media Analysis Engine," ACM Transactions on Intelligent Systems and Technology, 2025.
Jinfa Huang*, Jinsheng Pan*, Zhongwei Wan, Hanjia Lyu, Jiebo Luo, "Evolver: Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detection," International Conference on Computational Linguistics (COLING), Abu Dhabi, UAE, 2025.

2024

Fenglin Liu, Zheng Li, Qingyu Yin, Jinfa Huang, Xian Wu, Anshul Thakur, Kim Branson, Patrick Schwab, Bing Yin, Yefeng Zheng, Jiebo Luo, and David Clifton, "A Multimodal Multidomain Multilingual Medical Foundation Model for Zero Shot Clinical Diagnosis", npj Digital Medicine, in press.
Yongsheng Yu*, Ziyun Zeng*, Hang Hua, Jianlong Fu, Jiebo Luo, "PromptFix: You Prompt and We Fix the Photo," Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, December 2024.
Shenghai Yuan, Jinfa Huang, Yongqi Xu, YaoYang Liu, Shaofeng Zhang, Yujun Shi, Ruijie Zhu, Xinhua Cheng, Jiebo Luo, Li Yuan, "ChronoMagic: A Benchmark for Metamorphic Evaluation of Time-lapse Text-to-Video Generation," Dataset & Benchmark Track, Conference on Neural Information Processing Systems (NeurIPS), Vancouver, Canada, December 2024. (Spotlight)
Daoan Zhang*, Junming Yang*, Hanjia Lyu*, Zijian Jin, Yuan Yao, Mingkai Chen, and Jiebo Luo, "A Benchmark and Chain-of-Thought Prompting Strategy for Large Multimodal Models with Multiple Image Inputs," International Conference on Pattern Recognition (ICPR), Kolkata, India, December 2024.
Jiayi Hao, Yulin Feng, Wenbin Li, Jiebo Luo, "Rare Fungi Image Classification Based on Few-Shot Learning and Data Augmentation," International Conference on Pattern Recognition (ICPR), Kolkata, India, December 2024.
Jie An, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Lijuan Wang, Jiebo Luo, "OpenLEAF: A Novel Benchmark for Open-Domain Interleaved Image-Text Generation," Brave New Ideas Track, ACM Multimedia Conference, Melbourne, Australia, October 2024.
Zixuan Wang, Jiayi Li, Xiaoyu Qin, Shikun Sun, Songtao Zhou, Jia Jia, Jiebo Luo, "DanceCamAnimator: Keyframe-Based Controllable 3D Dance Camera Synthesis," ACM Multimedia Conference, Melbourne, Australia, October 2024.
Hang Hua, Jing Shi, Kushal Kafle, Simon Jenni, Daoan Zhang, John Collomosse, Scott Cohen, Jiebo Luo， "FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction," European Conference on Computer Vision (ECCV), Milan, Italy, September 2024.
Xinnong Zhang*, Haoyu Kuang*, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuanjing Huang, and Zhongyu Wei, "SoMeLVLM: A Large Vision Language Model for Social Media Processing," Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, August 2024.
Wenting Chen, Linlin Shen, Jingyang Lin, Jiebo Luo, Xiang Li, Yixuan Yuan, "Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation," Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, August 2024.
Junyu Chen, Jie An, Hanjia Lyu, Christopher Kanan, Jiebo Luo, "Holistic Visual-Textual Sentiment Analysis with Prior Models," IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, August 2024.
Yiwei Han, Kaiyi Qi, Jiebo Luo, "Plastic Surgery Image Classification and Generation," IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR), San Jose, CA, August 2024.
Jie An, Zhengyuan Yang, Jianfeng Wang, Linjie Li, Zicheng Liu, Lijuan Wang, Jiebo Luo, "Bring Metric Functions into Diffusion Models," 33rd International Joint Conference on Artificial Intelligence (IJCAI), Jeju, South Korea, August 2024.
Yuying Li, Zeyan Liu, Junyi Zhao, Liangqin Ren, Fengjun Li, Jiebo Luo, and Bo Luo, "The Adversarial AI-Art: Understanding, Generation, Detection, and Benchmarking," 29th European Symposium on Research in Computer Security (ESORICS), Bydgoszcz, Poland, September 2024.
Junyu Chen, Jie An, Hanjia Lyu, and Jiebo Luo, "Learning to Evaluate the Artness of AI-generated Images," IEEE Transactions on Multimedia, 26:10731-10740, 2024.
Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Eli Shechtman, Connelly Barnes, Jianming Zhang, Qing Liu, Yuqian Zhou, Sohrab Amirghodsi, Jiebo Luo, "Structure-Guided Image Completion with Image-level and Object-level Semantic Discriminators," IEEE Transactions on Pattern Analysis and Machine Intelligence, Early Access, 2024.
Shuo Yang, Zirui Zhang, Xinxiao Wu, Jiebo Luo, "Dynamic Pathway for Query-aware Feature Learning in Language-driven Action Localization," IEEE Transactions on Multimedia, Early Access, 2024.
Ali Vosoughi, Shijian Deng, Songyang Zhang, Yapeng Tian, Chenliang Xu, Jiebo Luo, "Cross Modality Bias in Visual Question Answering: A Causal View with Possible Worlds VQA," IEEE Transactions on Multimedia, Early Access, 2024.
Yuheng Shi, Xinxiao Wu. Jiebo Luo, "Knowledge Prompting for Few-shot Action Recognition in Videos," IEEE Transactions on Multimedia, Early Access, 2024.
Yujian Feng, Feng Chen, Yimu Ji, Fei Wu, Tianliang Liu, Shangdong Liu, Xiao-Yuan Jing, Jiebo Luo, "Cross-modality Spatial-temporal Transformer for Video-based Visible-infrared Person Re-identification, " IEEE Transactions on Multimedia, Early Access, 2024.
Peggy Tang, Kun Hu, Lei Zhang, Jiebo Luo, Zhiyong Wang, "TLDW: Extreme Multimodal Summarisation of News Videos," IEEE Transactions on Circuits and Systems for Video Technology, 34(3), March 2024.
Hanjia Lyu, Song Jiang, Hanqing Zeng, Yinglong Xia, Qifan Wang, Si Zhang, Ren Chen, Chris Leung, Jiajie Tang, and Jiebo Luo, "LLM-Rec: Personalized Recommendation via Prompting Large Language Models," Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, USA, June 2024.
Yuan Yao, Yuanhan Zhang, Zhenfei Yin, Jiebo Luo, Wanli Ouyang, "3D Point Cloud Pre-training with Knowledge Distillation from 2D Images," IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, Canada, July 2024.
Yongsheng Yu, Jiebo Luo, "Chain-of-Thought Prompting for Demographic Inference with Large Multimodal Models," IEEE International Conference on Multimedia and Expo (ICME), Niagara Falls, Canada, July 2024.
Zixuan Wang, Jia Jia, Shikun Sun, Haozhe Wu, Rong Han, Zhenyu Li, Di Tang, Zhou Jiaqing, Jiebo Luo, "DanceCamera3D: 3D Camera Movement Synthesis with Music and Dance," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2024.
Zhikai Chen, Fuchen Long, Zhaofan Qiu, Ting Yao, Wengang Zhou, Jiebo Luo, Tao Mei, "Learning Spatial Adaptation and Temporal Coherence in Diffusion Models for Video Super-Resolution," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2024.
Xinyi Mou, Zejun Li, Hanjia Lyu, Jiebo Luo, Zhongyu Wei, "Unifying Local and Global Knowledge: Empowering Large Language Models as Political Experts with Knowledge Graphs," The Web Conference (WWW), Singapore, May 2024.
Hanqing Zeng*, Hanjia Lyu*, Diyi Hu, Yinglong Xia, and Jiebo Luo, "Mixture of Weak & Strong Experts on Graphs," Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria, May 2024.
Shaofeng Zhang, Jinfa Huang, Qiang Zhou, Zhibin Wang, Fan Wang, Jiebo Luo, Junchi Yan, "Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach," Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria, May 2024.
Wenxi Yue, Jing Zhang, Kun Hu, Yong Xia, Jiebo Luo, Zhiyong Wang, "SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation," The 38th AAAI Conference on Artificial Intelligence (AAAI), February 2024.
Cong Jin, Ruolin Zhu, Zixing Zhu, Min Yang, Minghui Qiu, Jiebo Luo, "MtArtGPT: A Multi-task Art Generation System with Pre-Trained Transformer," IEEE Transactions on Circuits and Systems for Video Technology, Early Access, 2024.
Zhikai Chen, Fuchen Long, Zhaofan Qiu, Juyong Zhang, Zheng-jun Zha, Ting Yao, Jiebo Luo, "A Closer Look at the Reflection Formulation in Single Image Reflection Removal," IEEE Transactions on Image Processing, Early Access, 2024.

2023

Zhikai Chen, Fuchen Long, Zhaofan Qiu, Juyong Zhang, Zheng-jun Zha, Ting Yao, Jiebo Luo, "A Closer Look at the Reflection Formulation in Single Image Reflection Removal," IEEE Transactions on Image Processing, Early Access, 2023.
Jingyang Lin, Hang Hua, Ming Chen, Yikang Li, Mark Hsiao, Chiuman He, Jiebo Luo, "VideoXum: Cross-modal Visual and Textural Summarization of Videos," IEEE Transactions on Multimedia, Early Access. [Project Page]
Hang Hua, Xingjian Li, Dejing Dou, Cheng-Zhong Xu, Jiebo Luo, "Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization," IEEE Transactions on Neural Networks and Learning Systems, Early Access.
Mohammad Mahdi Kamani, Yuhang Yao, Hanjia Lyu, Zhongwei Cheng, Lin Chen, Liangju Li, Carlee Joe-Wong, Jiebo Luo, "Wyze Rule: Federated Rule Dataset for Rule Recommendation Benchmarking," Neural Information Processing Systems (NeurlPS), December 2023.
Alexander Martin, Haitian Zheng, Jie An, Jiebo Luo, "Jurassic World Remake: Bringing Ancient Fossils Back to Life via Zero-Shot Long Image-to-Image Translation," ACM Multimedia Conference, Brave New Ideas track, Ottawa, Canada, October 2023. (Oral Presentation)
Wenbin Li, Jing Huo, Yang Gao, Lei Wang, Jiebo Luo, "LibFewShot: A Comprehensive Library for Few-shot Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, Early Access.
Yushi Hu, Hang Hua, Zhengyuan Yang, Weijia Shi, Noah Smith, Jiebo Luo, "Prompt-Guided Image Captioning for VQA with GPT-3," International Conference on Computer Vision (ICCV), Paris, France, October 2023.
Pingyu Wu, Wei Zhai, Yang Cao, Jiebo Luo, Zheng-Jun Zha, "Spatial-Aware Token for Weakly Supervised Object Localization," International Conference on Computer Vision (ICCV), Paris, France, October 2023.
Yuhang Yang, Wei Zhai, Hongchen Luo, Yang Cao, Jiebo Luo, Zheng-Jun Zha, "Grounding 3D Object Affordance from 2D Interactions in Images," International Conference on Computer Vision (ICCV), Paris, France, October 2023.
Peggy Tang, Kun Hu, Lei Zhang, Jiebo Luo, Zhiyong Wang, "TL;DW: Extreme Multimodal Summarisation of News Videos," IEEE Transactions on Circuits and Systems for Video Technology, in press.
Siyu Huang, Jie An, Donglai Wei, Jiebo Luo, Hanspeter Pfister, "QuantArt: Quantizing Image Style Transfer Towards High Visual Fidelity," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
Zhikai Chen, Fuchen Long, Zhaofan Qiu, Ting Yao, Wengang Zhou, Jiebo Luo, Tao Mei, "AnchorFormer: Point Cloud Completion from Discriminative Nodes," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
Hongwei Xue, Peng Gao, Hongyang Li, Yu Qiao, Hao Sun, Houqiang Li, Jiebo Luo, "Stare at What You See: Masked Image Modeling without Reconstruction," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
Chengzhi Cao, Xueyang Fu, Hongjian Liu, Yukun Huang, Kunyu Wang, Jiebo Luo, Zheng-Jun Zha, "Event-guided Person Re-Identification via Sparse-Dense Complementary Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
Jin Chen, Zhi Gao, Xinxiao Wu, Jiebo Luo, "Meta-causal Learning for Single Domain Generalization," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Vancouver, Canada, June 2023.
Hongwei Xue, Yuchong Sun, Bei Liu, Jianlong Fu, Ruihua Song, Houqiang Li, Jiebo Luo, "CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Alignment," The Eleventh International Conference on Learning Representations (ICLR), Kigali, Rwanda, May 2023.
Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Jianming Zhang, Ning Xu, Jiebo Luo, "Semantic Layout Manipulation with High-resolution Sparse Attention," IEEE Transactions on Pattern Analysis and Machine Intelligence, March 2023.
Hao Wang, Zheng-Jun Zha, Liang Li, Xuejin Chen, Jiebo Luo, "Semantic and Relation Modulation for Audio-Visual Event Localization," IEEE Transactions on Pattern Analysis and Machine Intelligence, February 2023.
Jie An, Tao Li, Haozhi Huang, Jinwen Ma, and Jiebo Luo, "Is Bigger Always Better? An Empirical Study on Efficient Architectures for Style Transfer and Beyond," IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii, January 2023.

2022

Songyang Zhang, Linfeng Song, Lifeng Jin, Haitao Mi, Kun Xu, Dong Yu and Jiebo Luo, "Learning a Grammar Inducer from Massive Uncurated Instructional Videos," The Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, December 2022.
Haitian Zheng, Zhe Lin, Jingwan Lu, Scott Cohen, Eli Shechtman, Connelly Barnes, Jianming Zhang, Ning Xu, Sohrab Amirghodsi, Jiebo Luo, "Image Inpainting with Cascaded Modulation GAN and Object-Aware Training," European Conference on Computer Vision (ECCV), Tel-Aviv, October 2022.
Hanjia Lyu, Jiebo Luo, "Understanding Political Polarization via Jointly Modeling Users, Connections and Multimodal Contents on Heterogeneous Graphs," ACM International Conference on Multimedia (ACM MM), Lisbon, Portugal, October 2022.
Zhaoyi Wan, Dejia Xu, Jian Wang, and Jiebo Luo. "Cloud2Sketch: Augmenting Clouds with Imaginary Sketches," ACM International Conference on Multimedia (ACM MM), Lisbon, Portugal, October 2022.
Wei Zhu, Zihe Zheng, Haitian Zheng, Hanjia Lyu, Jiebo Luo, "Learning to Aggregate and Refine Noisy Labels for Visual Sentiment Analysis," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
Zhengyuan Yang, Jingen Liu, Jing Huang, Xiaodong He, Tao Mei, Chenliang Xu, Jiebo Luo, "Cross-modal Contrastive Distillation for Instructional Activity Anticipation," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
Wei Xiong, Ding Liu, Xiaohui Shen, Chen Fang, Jiebo Luo, "Unsupervised Low-light Image Enhancement with Decoupled Networks," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
Zhongjie Yu, Gaoang Wang, Lin Chen, Jiebo Luo, Sebastian Raschka, "When Few-Shot Learning Meets Video Object Detection," International Conference on Pattern Recognition (ICPR), Montreal, Quebec, Canada, August 2022.
Wei Zhu, Le Lu, Jing Xiao, Mei Han, Jiebo Luo, Adam Harrison, "Localized Adversarial Domain Generalization," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
Kai Zhu, Yang Cao, Wei Zhai, Jiebo Luo, Zheng-jun Zha, "Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
Fuchen Long, Zhaofu Qiu, Yingwei Pan, Ting Yao, Jiebo Luo, Tao Mei. "Stand-Alone Inter-Frame Attention in Video Models," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
Jing Shi, Ning Xu, Haitian Zheng, Alex Smith, Jiebo Luo, Chenliang Xu, "SpaceEdit: Learning a Unified Editing Space for Open-Domain Image Editing," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
Shaofei Cai, Liang Li, Xinzhen Han, Jiebo Luo, Zheng-Jun Zha, Qingming Huang, "Automatic Relation-aware Graph Network Proliferation," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.

2021

Sheng Liu, Zhongwei Cheng, Lin Chen, Jiebo Luo, Junsong Yuan, "A Compositional Model for Visual Relationship Grounding," IEEE Transactions on Image Processing, in press.
Songyang Zhang, Houwen Peng, Yijuan Lu, Jianlong Fu, Jiebo Luo, "Multi-Scale 2D Temporal Adjacent Networks for Moment Localization with Natural Language," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo, "Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training," Neural Information Processing Systems (NeurlPS), December 2021.
Wentian Zhao, Xinxiao Wu, Jiebo Luo, "Multi-modal Dependency Tree for Video Captioning," Neural Information Processing Systems (NeurlPS), December 2021.
Jian Ding, Nan Xue, Gui-Song Xia, Xiang Bai, Wen Yang, Michael Yang, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, Liangpei Zhang, "Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Litao Yu, Zhibin Li, Min Xu, Yongsheng Gao, Jiebo Luo, Jian Zhang, "Distribution-aware Margin Calibration for Semantic Segmentation in Images," International Journal of Computer Vision, in press.
Zhengyuan Yang, Songyang Zhang, Liwei Wang, Jiebo Luo, "SAT: 2D Semantics Assisted Training for 3D Visual Grounding," International Conference on Computer Vision (ICCV), 2021. (oral presentation)
Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, Jiebo Luo, "Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization," International Conference on Computer Vision (ICCV), 2021.
Heliang Zheng, Huan Yang, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo, "Learning Dual-Path Embedding Network for Degraded-Reference Image Quality Assessment," International Conference on Computer Vision (ICCV), 2021.
Jing Bi, Jiebo Luo, Chenliang Xu, "Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learning," International Conference on Computer Vision (ICCV), 2021. (oral presentation)
Mengshi Qi, Jie Qin, Di Huang, Zhiqiang Shen, Yi Yang, and Jiebo Luo, "Latent Memory-augmented Graph Transformer for Visual Storytelling," ACM Multimedia Conference, Chengdu, China, October 2021.
Hongwei Xue, Bei Liu, Huan Yang, Jianlong Fu, Houqiang Li, Jiebo Luo, "Learning Fine-Grained Motion Embedding for Landscape Animation," ACM Multimedia Conference, Chengdu, China, October 2021. (oral presentation)
Songyang Zhang, Linfeng Song, Lifeng Jin, Dong Yu, Jiebo Luo, "Video-aided Unsupervised Grammar Induction," Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, June 2021. (Best Long Paper)
Hang Hua, Xingjian Li, Dejin Dou, Chengzhong Xu, Jiebo Luo, "Noise Stability Regularization for Improving BERT Fine-tuning," Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Mexico City, Mexico, June 2021.
Zhengyuan Yang, Yijuan Lu, Xi Yin, Dinei Florencio, Lijuan Wang, Cha Zhang, Lei Zhang, Jiebo Luo, "TAP: Text-Aware Pre-training for Text-VQA and Text-Caption," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021. (oral presentation)
Jie An, Siyu Huang, Yibing Song, Dejing Dou, Wei Liu, Jiebo Luo, "ArtFlow: Unbiased Image Style Transfer via Reversible Neural Flows," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
Jing Wang, Jinhui Tang, Mingkun Yang, Xiang Bai, Jiebo Luo, "Improving OCR-based Image Captioning by Incorporating Geometrical Relationship," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
Hao Wang, Zheng-Jun Zha, Liang Li, Dong Liu, Jiebo Luo, "Structured Multi-Level Interaction Network for Video Moment Localization via Language Query," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
Kecheng Zheng, Wu Liu, Lingxiao He, Tao Mei, Jiebo Luo, Zheng-Jun Zha, "Group-aware Label Transfer for Domain Adaptive Person Re-identification, " IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2021.
Cheng Peng, Haofu Liao, Gina Wong, Jiebo Luo, S. Kevin Zhou, Rama Chellappa, "XraySyn: Realistic View Synthesis from a Single Radiograph through CT Prior," The 35th AAAI Conference on Artificial Intelligence (AAAI), February 2021.
Jin Chen, Xinxiao Wu, Yao Hu, Jiebo Luo, ''Spatial-temporal Causal Inference for Partial Image-to-video Adaptation," The 35th AAAI Conference on Artificial Intelligence (AAAI), February 2021.
Haitian Zheng, Lele Chen, Chenliang Xu, Jiebo Luo, "Pose Flow Learning from Person Images for Posed Guided Synthesis," IEEE Transactions on Image Processing, 30:1898-1909, 2021.
Yanlong Dong, Ying Zhang, Lin Ma, Zhi Wang, Jiebo Luo, ''Unsupervised text-to-image synthesis,'' Pattern Recognition, 110: 107573, 2021.

2020

Zhengyuan Yang, Tushar Kumar, Tinalang Chen, Jinsong Su, Jiebo Luo, ''Grounding-Tracking-Integration,'' IEEE Transactions on Circuits and Systems for Video Technology, in press.
Xinxiao Wu, Ruiqi Wang, Jingyi Hou, Hanxi Lin, Jiebo Luo, ''Spatial-Temporal Relation Reasoning for Action Prediction in Videos,'' International Journal of Computer Vision (IJCV), in press.
Jie An, Tianlang Chen, Songyang Zhang, Jiebo Luo, "Global Image Sentiment Transfer," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
Mingkun Yang, Haitian Zheng, Xiang Bai, Jiebo Luo, "Cost-Effective Adversarial Attacks against Scene Text Recognition," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
Gaoang Wang, Lin Chen, Mingwei He, Jiebo Luo, "DAIL: Dataset-Aware and Invariant Learning for Face Recognition," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
Weijian Li, Haofu Liao, Shun Miao, Le Lu, Jiebo Luo, "Unsupervised Learning of Landmarks based on Inter-Intra Subject Consistencies," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
Zhengyuan Yang, Yuncheng Li, Linjie Yang, Ning Zhang, Jiebo Luo, "Weakly Supervised Body Part Segmentation with Pose based Part Priors," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
Zhengyuan Yang, Amanda Kay, Yuncheng Li, Wendi Cross, Jiebo Luo, "Pose-based Body Language Recognition for Emotion and Psychiatric Symptom Interpretation," International Conference on Pattern Recognition (ICPR), Milan, Italy, December 2020.
Heliang Zheng, Jianlong Fu, Yanhong Zeng, Zheng-jun Zha, Jiebo Luo, "Learning Semantic-aware Normalization for Generative Adversarial Networks," Neural Information Processing Systems (NeurlPS), December 2020. (spotlight presentation)
Yiming Xu, Lin Chen, Zhongwei Cheng, Lixin Duan, Jiebo Luo, "Open-Ended Visual Question Answering by Multi-Modal Domain Adaptation," The Conference on Empirical Methods in Natural Language Processing (EMNLP), Findings of EMNLP, November 2020.
Xiankai Lu, Wenguan Wang, Jianbing Shen, David Crandall, Jiebo Luo, "Zero-Shot Video Object Segmentation with Co-Attention Siamese Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Wenbin Li*, Wei Xiong*, Haofu Liao, Jing Huo, Yang Gao, Jiebo Luo, "CariGAN: Caricature Generation through Weakly Paired Adversarial Learning," Neural Networks, in press.
Tianlang Chen, Wei Xiong, Haitian Zheng, Jiebo Luo, "Image Sentiment Transfer," Brave and New Ideas Track, ACM Multimedia Conference, Seattle, WA, October 2020. (Oral Presentation)
Jing Wang, Jinhui Tang, Jiebo Luo, "Multimodal Attention with Image Text Spatial Relationship for OCR-Based Image Captioning," ACM Multimedia Conference, Seattle, WA, October 2020.
Mengshi Qi, Jie Qin, Xiantong Zhen, Di Huang, Yi Yang, Jiebo Luo, "Few-Shot Ensemble Learning for Video Classification with SlowFast Memory Networks," ACM Multimedia Conference, Seattle, WA, October 2020.
Huan Lin, Fandong Meng, Jinsong Su, Yongjing Yin, Zhengyuan Yang, Yubin Ge, Jie Zhou, Jiebo Luo, "Dynamic Context-guided Capsule Network for Multimodal Machine Translation," ACM Multimedia Conference, Seattle, WA, October 2020.
Rui Zhao, Kecheng Zheng, Zheng-Jun Zha, Hongtao Xie, Jiebo Luo, "Memory Enhanced Embedding Learning for Cross-Modal Video-Text Retrieval," ACM Multimedia Conference, Seattle, WA, October 2020.
Yangchun Zhu, Zheng-Jun Zha, Tianzhu Zhang, Jiawei Liu, Jiebo Luo, "A Structured Graph Attention Network for Vehicle Re-Identification," ACM Multimedia Conference, Seattle, WA, October 2020. (Oral Presentation)
Hao Wang, Zheng-Jun Zha, Xuejin Chen, Zhiwei Xiong, Jiebo Luo, "Dual Path Interaction Network for Video Moment Localization," ACM Multimedia Conference, Seattle, WA, October 2020.
Haitian Zheng, Lele Chen, Chenliang Xu, Jiebo Luo, "Pose Flow Learning from Person Images for Posed Guided Synthesis," IEEE Transactions on Image Processing, in press.
Fuchen Long, Ting Yao, Zhaofan Qiu, Xinmei Tian, Jiebo Luo, Tao Mei, "Learning to Localize Actions from Moments," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020. (oral presentation)
Jianxin Lin, Yingxue Pang, Yingce Xia, Zhibo Chen, Jiebo Luo, "TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020. (spotlight presentation)
Zhengyuan Yang, Tianlang Chen, Liwei Wang, Jiebo Luo, "Improving One-stage Visual Grounding by Recursive Sub-query Construction," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
Tianlang Chen, Jiajun Deng, Jiebo Luo, "Adaptive Offline Quintuplet Loss for Image-text Matching," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
Haitian Zheng, Haofu Liao, Lele Chen, Wei Xiong, Tianlang Chen, Jiebo Luo, "Example-Guided Image Synthesis across Arbitrary Scenes using Masked Spatial-Channel Attention and Self-Supervision," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
Weijian Li, Yuhang Lu, Kang Zheng, Haofu Liao, Chihung Lin, Jiebo Luo, Chi-Tung Cheng, Jing Xiao, Le Lu, Chang-Fu Kuo, Shun Miao, "Cross-domain Structured Landmark Detection via Progressive Topology-Adapting Deep Graph Learning," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
Mang Ye, Jianbing Shen, David J. Crandall, Ling Shao, Jiebo Luo, "Dynamic Dual-Attentive Aggregation Learning for Visible-Infrared Person Re-Identification," European Conference on Computer Vision (ECCV), Glasgow, UK, August 2020.
Huafeng Li, Jiajia Xu, Zhengtao Yu, and Jiebo Luo, "Jointly Learning Commonality and Specificity Dictionaries for Person Re-Identification," IEEE Transactions on Image Processing, in IEEE Xplore.
Xin Liu, Kai Liu, Jinsong Su, Yebin Ge, Xiang Li, Bin Wang, Jiebo Luo, "An Iterative Multi-Source Mutual Knowledge Transfer Framework for Machine Reading Comprehension," International Joint Conference on Artificial Intelligence (IJCAI), Yokohama, Japan, July 2020.
Yongjing Yin, Fandong Meng, Jinsong Su, Chunlun Zhou, Zhengyuan Yang, Jie Zhou, Jiebo Luo, "A Novel Graph-based Multi-modal Fusion Encoder for Neural Machine Translation," Annual Meeting of the Association for Computational Linguistics (ACL), Seattle, WA, July 2020.
Mengshi Qi, Yunhong Wang, Annan Li, Jiebo Luo, "STC-GAN: Spatio-temporally coupled Generative Adversarial Networks for Predictive Scene Parsing," IEEE Transactions on Image Processing, 29: 5420-5430, 2020.
Jingyi Hou, Xinxiao Wu, Ruiqi Wang, Jiebo Luo, Yunde Jia, "Confidence-Guided Self Refinement for Action Prediction in Untrimmed Videos," IEEE Transactions on Image Processing, in press.
Wei Xiong, Yutong He, Yixuan Zhang, Wenhan Luo, Lin Ma, Jiebo Luo, "Fine-grained Image-to-Image Transformation towards Visual Recognition," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
Jie Chen, Zhiheng Li, Chenliang Xu, Jiebo Luo, "Learning a Weakly-Supervised Video Actor-Action Segmentation Model with a Wise Selection," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020. (oral presentation)
Zhongjie Yu, Lin Chen, Zhongwei Cheng, Jiebo Luo, "TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
Zhaoyi Wang, Jielei Zhang, Liang Zhang, Cong Yao, Jiebo Luo, "On Vocabulary Reliance in Scene Text Recognition," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
Jiamin Wu, Tianzhu Zhang, Zheng-Jun Zha, Jiebo Luo, Yongdong Zhang, Feng Wu, "Self-supervised Domain-aware Generative Network for Generalized Zero-shot Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.
Jie An, Haoyi Xiong, Jun Huan, Jiebo Luo, "Ultrafast Photorealistic Style Transfer via Neural Architecture Search," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020. (oral presentation)
Tianlang Chen, Jiebo Luo, "Expressing Objects just like Words: Recurrent Visual Embedding for Image-Text Matching," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
Songyang Zhang, Houwen Peng, Jianlong Fu, Jiebo Luo, "Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
Jinyi Hou, Xinxiao Wu, Yunde Jia, Jiebo Luo, "Joint Commonsense and Relation Reasoning for Image and Video Captioning," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
Jiali Zeng, Linfeng Song, Jinsong Su, Xie Jun, Wei Song, Jiebo Luo, "Neural Simile Recognition with Cyclic Multitask Learning and Local Attention," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
Yongjing Yin, Fandong Meng, Jinsong Su, Yubin Ge, Linfeng Song, Jie Zhou, Jiebo Luo, "Enhancing Pointer Network for Sentence Ordering with Pairwise Ordering Predictions," The 34th AAAI Conference on Artificial Intelligence (AAAI), New York, NY, February 2020.
Heliang Zheng, Zheng-Jun Zha, Jiebo Luo, "Learning Rich Part Hierarchies with Progressive Attention Networks for Fine-Grained Image Recognition," IEEE Transactions on Image Processing, 29: 476-488, 2020.
Tianliang Liu, Quanzeng You, Yanzhang Wang, Xiaodong Dong, Xiubin Dai, Jiebo Luo, "Sentiment Recognition for Short Annotated GIFs Using Visual-Textual Fusion," IEEE Transactions on Multimedia, 22(4): 1098-1110, 2020.

2019

Jianxin Lin, Yingce Xia, Sen Liu, Tao Qin, Zhibo Chen, Jiebo Luo, "Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo, "Learning Deep Bilinear Transformation for Fine-grained Image Representation," Neural Information Processing Systems (NeurlPS), Vancouver, Canada, December 2019.
Jaili Zeng, Yang Liu, Jinsong Su, Yubing Ge, Yaojie Lu, Yongjing Yin and Jiebo Luo, "Iterative Dual Domain Adaptation for Neural Machine Translation," Conference on Empirical Methods in Natural Language Processing (EMNLP), Hong Kong, November 2019.
Biao Zhang, Deyi Xiong, Jinsong Su, and Jiebo Luo, "Future-Aware Knowledge Distillation for Neural Machine Translation," IEEE Transactions on Audio, Speech and Language Processing, in press.
Fuchen Long, Ting Yao, Xinwei Tian, Tao Mei, Jiebo Luo, "Coarse-to-Fine Localization of Temporal Action Proposals," IEEE Transactions on Multimedia, in press.
Rongrong Ji, Ke Li, Feng Guo, Xiaoshuai Sun, Yan Wang, Feiyue Huang, Jiebo Luo, Gary Huang, "Semi-Supervised Adversarial Monocular Depth Estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Yang Cong, Baojie Fan, Dongdong Hou, Huijie Fan, Kaizhou Liu, Jiebo Luo, "Novel Event Analysis for Human-Machine Collaborative Underwater Exploration," Pattern Recognition, in press.
Zhengyuan Yang, Boqing Gong, Liwei Wang, Wenbing Huang, Dong Yu, Jiebo Luo, "A Fast and Accurate One-Stage Approach to Visual Grounding," International Conference on Computer Vision (ICCV), Seoul, South Korea, October 2019. (oral presentation)
Tianlang Chen, Zhaowen Wang, Ning Xu, Hailin Jin, Jiebo Luo. "Large-scale Tag-based Font Retrieval with Generative Feature Learning," International Conference on Computer Vision (ICCV), Seoul, South Korea, October 2019.
Jingyi Hou, Xinxiao Wu, Wentian Zhao, Jiebo Luo, "Joint Syntax Representation Learning and Visual Cues Translation for Video Captioning," International Conference on Computer Vision (ICCV), Seoul, South Korea, October 2019.
Songyang Zhang, Jinsong Su, Jiebo Luo, "Exploiting Temporal Relationships in Video Moment Localization with natural Language," ACM Multimedia Conference, Nice, France, October 2019.
Jialong Tang, Ziyao Lu, Jinsong Su, Yubin Ge, Linfeng Song, Le Sun, and Jiebo Luo, "Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis," Annual Meeting of the Association for Computational Linguistics (ACL), Florence, Italy, July 2019.
Yongjing Yin, Linfeng Song, Jinsong Su, Jiali Zeng, Chulun Zhou, and Jiebo Luo, "Graph-based Neural Sentence Ordering," International Joint Conference on Artificial Intelligence (IJCAI), Macau, July 2019.
Heliang Zheng, Zheng-Jun Zha, Jiebo Luo, "Learning Rich Part Hierarchies with Progressive Attention Networks for Fine-Grained Image Recognition," IEEE Transactions on Image Processing, in press.
Yang Feng, Lin Ma, Wei Liu, Jiebo Luo, "Unsupervised Image Captioning," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019. [PDF] [Code]
Yang Feng, Lin Ma, Wei Liu, Jiebo Luo, "Spatio-temporal Video Re-localization by Warp LSTM," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019. [PDF] [Code]
Wei Xiong, Jiahui Yu, Zhe Lin, Jimei Yang, Xin Lu, Connelly Barnes, Jiebo Luo, "Foreground-aware Image Inpainting," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019. [Project Page]
Mengshi Qi*, Weijian Li*, Zhengyuan Yang, Yunhong Wang, Jiebo Luo, "Attentive Relational Networks for Mapping Images to Scene Graphs," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019.
Heliang Zheng, Jianlong Fu, Zheng-Jun Zha, Jiebo Luo, "Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019.
Fuchen Long, Ting Yao, Xinmei Tian, Jiebo Luo, Tao Mei, "Gaussian Temporal Awareness Networks for Action Localization", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019. (oral presentation)
Wei-An Lin*, Haofu Liao*, Cheng Peng, Xiaohang Sun, Jingdan Zhang, Jiebo Luo, Rama Chellappa, S. Kevin Zhou, "DuDoNet: Dual Domain Network for CT Metal Artifact Reduction," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019.
Haofu Liao, Wei-An Lin, Jiarui Zhang, Jingdan Zhang, Jiebo Luo, S. Kevin Zhou, "Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019.
Jingyuan Chen, Lin Ma, Wei Liu, Jiebo Luo, "Localizing Natural Language in Videos," The 33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, February 2019.

2018

Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, Jiebo Luo, "Tell-and-Answer: Towards Explainable Visual Question Answering using Attributes and Captions," Conference on Empirical Methods in Natural Language Processing (EMNLP), Brussels, Belgium, November 2018. (oral presentation)
Mingkun Yang, Yongchao Xu, Xiang Bai, Jiebo Luo, "Integrating Scene Text and Visual Appearance for Fine-Grained Image Classification," IEEE Access, in IEEE Xplore.
Zhengyuan Yang, Yuncheng Li, Jianchao Yang, and Jiebo Luo, "Action Recognition with Spatio-Temporal Visual Attention on Skeleton Image Sequences," IEEE Transactions on Circuits and Systems for Video Technology, in IEEE Xplore.
Wenbin Li, Jing Huo, Yinghuan Shi, Yang Guo, Lei Wang, Jiebo Luo, "A Joint Local and Global Deep Learning Method for Caricature Recognition," Asian Conference on Computer Vision (ACCV), Perth, Australia, December 2018.
Haofu Liao, Gareth Funka-Lea, Yefeng Zheng, Kevin Zhou, Jiebo Luo, "Face Completion with Semantic Knowledge and Collaborative Adversarial Learning," Asian Conference on Computer Vision (ACCV), Perth, Australia, December 2018.
Tianlang Chen, Zhongping Zhang, Quanzeng You, Chen Fang, Zhaowen Wang, Hailin Jin, Jiebo Luo, "Factual or Emotional: Stylized Image Captioning with Adaptive Learning and Attention," European Conference on Computer Vision (ECCV), Munich, Germany, September 2018.
Yang Feng, Lin Ma, Wei Liu, Tong Zhang, Jiebo Luo, "Video Re-localization via Cross Gated Bilinear Matching," European Conference on Computer Vision (ECCV), Munich, Germany, September 2018. [PDF][Github]
Qing Li, Qingyi Tao, Shafiq Joty, Jianfei Cai, Jiebo Luo, "VQA-E: Explaining, Elaborating, and Enhancing Your Answers for Visual Questions," European Conference on Computer Vision (ECCV), Munich, Germany, September 2018.
Mengshi Qi, Jie Qin, Annan Li, Yunhong Wang, Jiebo Luo, Luc Van Gool, "stagNet: An Attentive Semantic RNN for Group Activity Recognition," European Conference on Computer Vision (ECCV), Munich, Germany, September 2018.
Fuchen Long, Ting Yao, Tao Mei, Jiebo Luo, "Deep Domain Adaptation Hashing with Adversarial Learning," ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), Ann Arbor, MI, July 2018. (Oral presentation）
Xiaobai Liu, Qian Xu, Jingjie Yang, Jacob Thalman, Shuicheng Yan, and Jiebo Luo, "Learning Multi-Instance Deep Ranking and Regression Network for Visual House Appraisal," IEEE Transactions on Knowledge and Data Engineering (TKDE) 30(8): 1496-1506, 2018.
Zhengyuan Yang, Yixuan Zhang, Jerry Yu, Junjie Cai, Jiebo Luo, "End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception," IAPR/IEEE International Conference on Pattern Recognition (ICPR), Beijing, China, August 2018. (Best Industrial Related Paper）[PDF][Demo]
Zhengyuan Yang, Yuncheng Li, Jianchao Yang, Jiebo Luo, "Action Recognition with Visual Attention on Skeleton Images," IAPR/IEEE International Conference on Pattern Recognition (ICPR), Beijing, China, August 2018. (Oral presentation）
Zhongping Zhang, Yixuan Zhang, Zheng Zhou, Jiebo Luo, "Boundary-based Image Forgery Detection by Fast Shallow CNN," IAPR/IEEE International Conference on Pattern Recognition (ICPR), Beijing, China, August 2018.
Viet-Duy Nguyen, Minh Tran, Jiebo Luo, "Are French Really That Different? Recognizing Europeans from Faces Using Data-Driven Learning," IAPR/IEEE International Conference on Pattern Recognition (ICPR), Beijing, China, August 2018.
Quanzeng You, Zhengyou Zhang, Jiebo Luo, "End-to-end Convolutional Semantic Embeddings," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, June 2018.
Wei Xiong, Wenhan Luo, Lin Ma, Wei Liu, Jiebo Luo, "Learning to Generate Time-Lapse Videos Using Multi-Stage Dynamic Generative Adversarial Networks," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, June 2018.
Danna Gurari, Qing Li, Abigale Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, Jeffery Bigham, "VizWiz Grand Challenge: Answering Visual Questions from Blind People," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, June 2018. (spotlight presentation)
Gui-Song Xia, Xiang Bai, Jian Ding, Serge Belongie, Jiebo Luo, Mihai Datcu, Marcello Pelillo, Liangpei Zhang, "DOTA: A Large-scale Dataset for Object Detection in Aerial Images," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, June 2018.
Xinpeng Chen, Jingyuan Chen, Lin Ma, Jian Yao, Wei Liu, Jiebo Luo and Tong Zhang, "Fine-grained Video Attractiveness Prediction Using Multimodal Deep Learning on a Large Real-world Dataset," The Web Conference (WWW), Lyon, France, April 2018.
Yu Wang, Haofu Liao, Yang Feng, Xiangyang Xu, Jiebo Luo, "Do They All Look the Same? Deciphering Chinese, Japanese and Koreans by Fine-Grained Deep Learning," IEEE International Conference on Multimedia Information Processing and Retrieval, Miami, FL, April 2018.
Tianlang Chen, Chenliang Xu, Jiebo Luo, "Improving Text-based Person Search by Spatial Matching and Adaptive Threshold," Winter Conference on Computer Vision (WACV), Lake Tahoe, NV, March 2018.
Xitong Yang, Jiebo Luo, "Towards Perceptual Image Dehazing by Physics-based Disentanglement and Adversarial Training", The 32nd AAAI Conference on Artificial Intelligence (AAAI), New Orleans, LA, February 2018.
Jingyi Hou, Xinxiao Wu, Jiebo Luo, "Unsupervised Deep Learning of Mid-Level Video Representation for Action Recognition," The 32nd AAAI Conference on Artificial Intelligence (AAAI), New Orleans, LA, February 2018.
Yang Cong, Ji Liu, Jiebo Luo, "User Attribute Discovery with Missing Labels," Pattern Recognition 73: 33-46, 2018.

2017

Heliang Zheng, Jianlong Fu, Tao Mei, and Jiebo Luo, "Learning Multi-Attention Convolutional Neural Network for Fine-Grained Image Recognition," International Conference on Computer Vision (ICCV), Venice, Italy, October 2017. (oral presentation)
Xiangyang Xu, Yuncheng Li, Gangshan Wu and Jiebo Luo, "Multi-modal Deep Feature Learning for RGB-D Object Detection," Pattern Recognition 72: 300-313, 2017.
Zhengyuan Yang, Wendi Cross, Jiebo Luo, "Personalized pose estimation for body language understanding," International Conference on Image Processing (ICIP), Beijing, China, September 2017. (oral presentation)
Honglin Zheng, Tianlang Chen, Quanzeng You, Jiebo Luo, "When Saliency Meets Sentiment: Understanding How Image Content Invokes Emotion and Sentiment," International Conference on Image Processing (ICIP), Beijing, China, September 2017. (oral presentation)
Lifang Wu, Shuang Liu, Meng Jian, Jiebo Luo, Xiuzhen Zhang, Mingchao Qi, "Reducing Noisy Labels on Weakly Labeled Data for Visual Sentiment Analysis," International Conference on Image Processing (ICIP), Beijing, China, September 2017.
Yuncheng Li, Liangliang Cao, Jiebo Luo, "Mining Fashion Outfit Composition Using an End-to-End Deep Learning Approach on Set Data," IEEE Trans. on Multimedia, 19(8): 1946-1955(2017).
Yanhao Zhang, Lei Qin, Rongrong Ji, Qingming Huang, Jiebo Luo, "Exploring Coherent Motion Patterns via Structured Trajectory Learning for Crowd Mood Modeling," IEEE Transactions on Circuits and Systems for Video Technology, 27(3): 635-648 (2017).
Fei Wu, Zhuhao Wang, Weiming Lu, Xi Li, Yi Yang, Jiebo Luo, Yueting Zhuang, "Regularized Deep Belief Network for Image Attribute Detection," IEEE Transactions on Circuits and Systems for Video Technology, 27(7): 1464-1477, 2017.
Qing Li, Zhaofan Qiu, Ting Yao, Tao Mei, Yong Rui, Jiebo Luo, "Learning hierarchical video representation for action recognition," International Journal of Multimedia Information Retrieval, 6(1): 85-98, 2017.
Quanzeng You, Liangliang Cao, Jiebo Luo, "Image Based Appraisal of Real Estate Properties," IEEE Transactions on. Multimedia, in press.
Xitong Yang, Sriganesh Madhvanath, Edgar A. Bernal, Palghat Ramesh, Radha Chitta, Jiebo Luo, "Deep Multimodal Representation Learning from Temporal Data," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, July 2017.
Marko Stamenovic, Jiebo Luo, "Machine Identification of High Impact Research through Text and Image Analysis," IEEE Big Multimedia Conference, Laguna Hills, CA, April 2017.
Quanzeng You, Hailin Jin, Jiebo Luo, "Visual Sentiment Analysis by Attending on Local Image Regions", The 31st AAAI Conference on Artificial Intelligence (AAAI), San Francisco, CA, February 2017.
Songhe Feng, Congyan Lang, Jiashi Feng and Jiebo Luo, "Human Facial Age Estimation by Cost-Sensitive Label Ranking and Trace Norm Regularization," IEEE Trans. on Multimedia, 19(1): 136-148, 2017.
Yang Cong, Ji Liu, Gan Sun, Quanzheng You, Yuncheng Li and Jiebo Luo, "Adaptive Greedy Dictionary Selection for Web Media Summarization," IEEE Trans. on Image Processing, 26(1): 185-195, 2017.

2016

Yang Feng, Yuncheng Li, Jiebo Luo, "Learning Effective Gait Features Using LSTM", International Conference on Pattern Recognition (ICPR), Cancun, Mexico, December 2016.
Iftekhar Naim, Abdullah Al Mamun, Young Chol Song, Jiebo Luo, Henry Kautz, Daniel Gildea, "Aligning Movies with Scripts by Exploiting Temporal Ordering Constraints", International Conference on Pattern Recognition (ICPR), Cancun, Mexico, December 2016.
Quanzeng You, Liangliang Cao, Hailin Jin, Jiebo Luo, "Robust Visual-Textual Sentiment Analysis: When Attention meets Tree-structured Recursive Neural Networks", ACM Multimedia Conference, Amsterdam, The Netherlands, October 2016.
Qing Li, Zhaofan Qiu, Ting Yao, Tao Mei, Yong Rui and Jiebo Luo, "Action Recognition by Learning Deep Multi-Granular Spatio-Temporal Video Representation," ACM/IAPR International Conference on Multimedia Retrieval (ICMR), New York City, June 2016. (Long paper, Best Paper Candidate)
Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, Jiebo Luo, "Image Captioning with Semantic Attention," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, June 2016. (spotlight presentation)
Yuncheng Li, Yale Song, Liangliang Cao, Joel Tetreault, Larry Goldberg, Jiebo Luo, "TGIF: A New Dataset and Benchmark on Animated GIF Description," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, June 2016. (spotlight presentation)
Tianliang Liu, Xincheng Wang, Xiubin Dai, Jiebo Luo, "Deep Recursive and Hierarchical Conditional Random Fields for Human Action Recognition," IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, NY, March 2016.
Liang Lin, Keze Wang, Wangmeng Zuo, Meng Wang, Lei Zhang, Jiebo Luo, "Deep Reconfigurable Models with Radius-Margin Bound for 3D Human Activity Recognition," International Journal of Computer Vision, available online.
Xinyang Cai, Wengang Zhou, Houqiang Li, Jiebo Luo, Lei Wu, "Effective Kinetic Skeleton Representation for Low Latency Human Action Recognition," IEEE Trans. Multimedia, available online.
Quanzeng You, Hailin Jin, Jianchao Yang, Jiebo Luo, "Cross-modality Consistent Regression for Joint Visual-Textual Sentiment Analysis," ACM International Conference on Web Search and Data Ming (WSDM), San Francisco, CA, February 2016. [PrePrintPDF]
Quanzeng You, Hailin Jin, Jianchao Yang, Jiebo Luo, "Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark," The 30th AAAI Conference on Artificial Intelligence (AAAI), Phoenix, AZ, January 2016. [PDF][Dataset Page]

2015

Yuncheng Li, Xitong Yang, Jiebo Luo, "Semantic Video Entity Linking based on Visual Content and Metadata," International Conference on Computer Vision (ICCV), Santiago, Chile, December 2015. [PrePrintPDF]
Yuncheng Li, Jifei Huang, Jiebo Luo, "Using User Generated Online Photos to Estimate and Monitor Air Pollution in Major Cities," ACM International Conference on Internet Multimedia Computing and Service (ICIMCS), August 2015. (Best Paper) [PDF] [Project Page]
Yuning Jiang, Jingjing Meng, Junsong Yuan, Jiebo Luo, "Randomized Spatial Context for Object Search," IEEE Trans. on Image Processing, 24(6): 1748-1762, 2015.
Wu Liu, Tao Mei, Yongdong Zhang, Cherry Che and Jiebo Luo, "Multi-Task Deep Visual-Semantic Embedding for Video Thumbnail Selection," IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, June 2015.
Wanying Ding, Junhuan Zhu, Lifan Guo, Xiaohua Hu, Jiebo Luo, Haohong Wang, "Jointly Image Topic and Emotion Detection using Multi-Modal Hierarchical Latent Dirichlet Allocation," Journal of Multimedia Information System, 2015.
Yang Cong, Ji Liu, Jiebo Luo, "Speeded up Low Rank Online Metric Learning for Object Tracking," IEEE Transactions on Circuits and Systems for Video Technology, 2015.

2010-2014

Young Chol Song, Henry Kautz, James Allen, Mary Swift, Yuncheng Li, Jiebo Luo, "A Markov Logic Framework for Recognizing Complex Events from Multimodal Data," ACM International Conference on Multimodal Interaction (ICMI), Sidney, Australia, December 2013.
Yao Zhou and Jiebo Luo, "A Practical Method for Counting Arbitrary Target Objects in an Arbitrary Scene," IEEE ICME, July 2013.
Congyan Lang, Jiashi Feng, Guangcan Liu, Jinghui Tang, Shuicheng Yan and Jiebo Luo, "Improving Bottom-up Saliency Detection by Looking into Neighbors," IEEE Transactions on Circuits and Systems for Video Technology, 23(6): 1016-1028, June 2013.
Minwoo Park, Jiebo Luo, Andrew Gallagher, Majid Rabbani, "Learning to Produce 3D Media from a Captured 2D Video," IEEE Transactions on Multimedia, 15(7): 1569-1578, 2013.
Hua Wang, Dhiraj Joshi, Jiebo Luo, and Heng Huang, "Simultaneous Image Annotation and Geo-Tag Prediction via Correlation Guided Structured Multi-Task Learning," IEEE Symposium on Multimedia (ISM), December 2012.
Jianchao Yang, Jiebo Luo, Jie Yu, Thomas. Huang, "Photo Stream Alignment and Summarization for Collaborative Photo Collection and Sharing," IEEE Transactions on Multimedia, 14(6): 1642-1651, December 2012.
Youjie Zhou, Jiebo Luo, " Geo-Location Inference on News Articles via Multimodal pLSA," ACM Multimedia Conference, Nara, Japan, October 2012.
Xiaobin Xu, Tao Mei, Wenjun Zeng, Nenghai Yu, Jiebo Luo, "AMIGO: Accurate Mobile Image GeOtagging," ACM International Conference on Internet Multimedia Computing and Services, Wuhan, China, September 2012. (Best Paper)
Lin Chen, Dong Xu, Ivor Tsang, and Jiebo Luo for the paper entitled, "Tag-Based Image Retrieval Improved by Augmented Features and Group-Based Refinement", IEEE Transactions on Multimedia, Volume 14, Number 4, August 2012 (2014 IEEE Multimedia Prize Paper Award).
Minwoo Park, Jiebo Luo, Andrew Gallagher, "Towards Assessing and Improving the Quality of Stereo Images," Special Issue on Emerging Techniques in 3D, IEEE Journal of Selected Topics in Signal Processing, 6(5): 460 - 470, 2012.
Siyu Xia, Ming Shao, Jiebo Luo, and Yun Fu, "Understanding Kin Relationships in a Photo," Special Issue on Learning Semantics from Multimedia Web Resources, IEEE Transactions on Multimedia, 14(4): 1046-1056, 2012.
Lin Chen, Dong Xu, Ivor W. Tsang, Jiebo Luo, "Tag-based Image Retrieval Improved by SVM with Augmented Features and Group-based Refinement," Special Issue on Learning Semantics from Multimedia Web Resources, IEEE Transactions on Multimedia, 14(4): 1057-1067, 2012.
Cong Yang, Junsong Yuan, Jiebo Luo, "Towards Scalable Summarization of Consumer Videos via Sparse Dictionary Selection," Special Issue on Object and Event Classification in Large-Scale Video Collections, IEEE Transactions on Multimedia, 14(1): 66-75, February 2012.
Dhiraj Joshi, Ritendra Datta, Elena Fedorovskaya, Jia Li, James Z. Wang, Jiebo Luo, "Computational inference of aesthetics, mood, and emotion in images," IEEE Signal Processing Magazine, 28(5): 94-115, September 2011.
Yiming Liu, Dong Xu, Ivor Tsang, Jiebo Luo, "Textual query of personal photos facilitated by large-scale web data," IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(5): 1022-1036, May 2011.
Minwoo Park, Jiebo Luo, Robert Collions, Yanxi Liu, Beyond GPS: Determining the viewing direction of a geotagged image," ACM Multimedia Conference, Firenze, Italy, October 2010.
Xiaobai Liu, Shuicheng Yan, Jiebo Luo, Jinhui Tang, ZhongYang Huang, Hai Jin, "Nonparametric Label-to-Region by Search," CVPR, 2010.
Jingen Liu, Jiebo Luo, Mubarak Shah, "Recognizing Realistic Actions from Videos in the Wild," IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, June 2009. (Oral Presentation) [Project Page]

machine learning

2024

Junda Wang, Weijian Li, Han Wang, Hanjia Lyu, Caroline Thirukumaran, Addisu Mesfin, Hong Yu, Jiebo Luo, "CRTRE: Causal Rule Generation with Target Trial Emulation Framework," IEEE Big Data Conference, Washington DC, December 2024.
Xinyi Mou, Zejun Li, Hanjia Lyu, Jiebo Luo, Zhongyu Wei, "Unifying Local and Global Knowledge: Empowering Large Language Models as Political Experts with Knowledge Graphs," The Web Conference (WWW), Singapore, May 2024.
Beichen Zhang, Liang Li, Zheng-Jun Zha, Jiebo Luo, Qingming Huang, "Downstream-Pretext Domain Knowledge Traceback for Active Learning," IEEE Transactions on Multimedia, 26:10585-10596, 2024.
Hanqing Zeng*, Hanjia Lyu*, Diyi Hu, Yinglong Xia, and Jiebo Luo, "Mixture of Weak & Strong Experts on Graphs," Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria, May 2024.
Jian Kang, Yinglong Xia, Ross Maciejewski, Jiebo Luo, Hanghang Tong, "Deceptive Fairness Attacks on Graphs via Meta Learning," Twelfth International Conference on Learning Representations (ICLR), Vienna, Austria, May 2024.
Jiamin Wu, Tianzhu Zhang, Zheng-Jun Zha, Jiebo Luo, Yongdong Zhang, Feng Wu, "Prototype-augmented Self-supervised Generative Network for Generalized Zero-shot Learning," IEEE Transactions on Image Processing, Early Access, 2024.

2023

Hang Hua, Xingjian Li, Dejing Dou, Cheng-Zhong Xu, Jiebo Luo, "Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization," IEEE Transactions on Neural Networks and Learning Systems, Early Access, 2023.
Wenbin Li, Jing Huo, Yang Gao, Lei Wang, Jiebo Luo, "LibFewShot: A Comprehensive Library for Few-shot Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, Early Access, 2023.
Wei Zhu, Weijian Li, Ray Dorsey, and Jiebo Luo, "Unsupervised Anomaly Detection by Densely Contrastive Learning for Time Series Data, " Neural Network, 2023.
Yuan Zhou, Yanrong Guo, Shijie Hao, Richang Hong, Jiebo Luo, "Few-shot Partial Multi-view Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Zhishuai Guo, Rong Jin, Jiebo Luo, Tianbao Yang, "FeDXL: Provable Federated Learning for Deep X-Risk Optimization, " International Conference on Machine Learning (ICML), Hawaii, July 2023.
Xiao Wang, Jingen Liu, Tao Mei, Jiebo Luo, "CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation," IEEE Transactions on Neural Networks and Learning Systems, Early Access, 2023.

2022

Wenbin Li, Yanfang Liu, Jing Huo, Yinghuan Shi, Yang Gao, Lei Wang, Jiebo Luo, "A Multilayer Framework for Online Metric Learning," IEEE Transactions on Neural Networks and Learning Systems, Early Access, 2022.
Wenbin Li, Lei Wang, Xingxing Zhang, Lei Qi, Jing Huo, Yang Gao, Jiebo Luo, "Defensive Few-shot Learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, Early Access, 2022.
W. Zhu, D. Song, Y. Chen, W. Cheng, B. Zong, T. Mizoguchi, C. Lumezanu, H. Chen, and J. Luo, "Deep Federated Anomaly Detection for Multivariate Time Series Data," IEEE International Conference on Big Data, Osaka, Japan, December 2022.
Yian Wang, Jian Kang, Yinglong Xia, Jiebo Luo, and Hanghang Tong, "iFiG: Individually Fair Multi-view Graph Clustering," IEEE International Conference on Big Data, Osaka, Japan, December 2022.
Wei Zhu, Jiebo Luo, "Federated Medical Image Analysis with Virtual Sample Synthesis," International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Singapore, September 2022.
Wei Zhu, Andrew White, and Jiebo Luo, "Federated Learning of Molecular Properties in a Heterogeneous Setting," Cell Patterns, 2022.
Wei Zhu, Le Lu, Jing Xiao, Mei Han, Jiebo Luo, Adam Harrison, "Localized Adversarial Domain Generalization," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
Kai Zhu, Yang Cao, Wei Zhai, Jiebo Luo, Zheng-jun Zha, "Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
Shaofei Cai, Liang Li, Xinzhen Han, Jiebo Luo, Zheng-Jun Zha, Qingming Huang, "Automatic Relation-aware Graph Network Proliferation," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), June 2022.
Jian Kang, Yan Zhu, Yinglong Xia, Jiebo Luo, Hanghang Tong. "RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network," ACM Web Conference (WWW), 2022.

2021

Hongwei Xue, Yupan Huang, Bei Liu, Houwen Peng, Jianlong Fu, Houqiang Li, Jiebo Luo, "Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training," Neural Information Processing Systems (NeurlPS), December 2021.
Yiming Xu, Lin Chen, Lixin Duan, Ivor Tsang, Jiebo Luo, "Open Set Domain Adaptation with Soft Unknown-Class Rejection," IEEE Transactions on Neural Networks and Learning Systems (TNNLS), in press.
Wei Zhu, Haitian Zheng, Haofu Liao, Weijian Li, Jiebo Luo, "Learning Bias-Invariant Representation by Cross-Sample Mutual Information Minimization," International Conference on Computer Vision (ICCV), 2021.
Guo-Jun Qi, Jiebo Luo, "Small Data Challenges in Big Data Era: A Survey of Recent Progress on Unsupervised and Semi-Supervised Methods," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Xiao Wang, Guo-Jun Qi, Jiebo Luo, "EnAET: A Self-Trained Framework for Semi-Supervised and Supervised Learning with Ensemble Transformation," IEEE Transactions on Image Processing, 30: 1639-1647, 2021.
Wei Zhu, Wenbin Li, Haofu Liao, and Jiebo Luo, ''Temperature Network for Few-shot Learning with Distribution-aware Large-margin Metric,'' Pattern Recognition, in press.

2020

Mengshi Qi, Jie Qin, Xiantong Zhen, Di Huang, Yi Yang, Jiebo Luo, "Few-Shot Ensemble Learning for Video Classification with SlowFast Memory Networks," ACM Multimedia Conference, Seattle, WA, October 2020.
Wenbin Li, Lei Wang, Jing Huo. Yinghuan Shi, Yang Gao, Jiebo Luo, "Asymmetric Distribution Measure for Few-shot Learning," International Joint Conference on Artificial Intelligence (IJCAI), Yokohama, Japan, July 2020.
Zhongjie Yu, Lin Chen, Zhongwei Cheng, Jiebo Luo, "TransMatch: A Transfer-Learning Scheme for Semi-Supervised Few-Shot Learning," IEEE/CVF Conferences on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020.

2019

Yi Wang, Nan Xue, Xin Fan, Jiebo Luo, Risheng Liu, Haojie Li, Bin Chen, Zhongxun Luo, "Novelty Detection and Online Learning for Chunk Data Streams," IEEE Transactions on Pattern Analysis and Machine Intelligence, in press.
Wenbin Li, Lei Wang, Jinglin Xu, Jing Huo, Yang Gao and Jiebo Luo, "Revisiting Local Descriptor based Image-to-Class Measure for Few-shot Learning," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019. [PDF][Github]
Wenbin Li, Jing Huo, Yinghuan Shi, Yang Guo, Lei Wang, Jiebo Luo, "Distribution Consistency based Covariance Metric Networks for Few Shot Learning," The 33rd AAAI Conference on Artificial Intelligence (AAAI), Honolulu, HI, February 2019. (oral presentation) [PDF][Github]
Liheng Zhang, Guo-Jun Qi, Liqiang Wang, Jiebo Luo, "AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Data," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019. (oral presentation)

2018

Yi Wang, Nan Xue, Xin Fan, Jiebo Luo, Risheng Liu, Haojie Li, Bin Chen, Zhongxun Luo, "Fast factorization-free Kernel Learning for Unlabeled Chunk Data Streams," International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, July 2018.
Xiaotong Zhang, Xianchao Zhang, Han Liu, Jiebo Luo, "Multi-Task Clustering with Model Relation Learning," International Joint Conference on Artificial Intelligence (IJCAI), Stockholm, Sweden, July 2018.
Yang Cong, Ji Liu, Jiebo Luo, "Sparse Low-rank Online Similarity Learning," IEEE Transactions on Cybernetics, 73: 33-46, 2018.
Yang Cong, Ji Liu, Jiebo Luo, "Online Similarity Learning for Big Data with Overfitting," IEEE Transactions on Big Data, 4(1): 78-89, 2018.

2017

Jianbo Yuan, Han Guo, Zhiwei Jin, and Jiebo Luo, "One-shot Learning for Fine-grained Relation Extraction via Convolutional Siamese Neural Network," IEEE Big Data Conference, Boston, MA, December 2017.
Yuncheng Li, Jianchao Yang, Yale Song, Liangliang Cao, Jia Li and Jiebo Luo, "Learning from Noisy Labels with Distillation," International Conference on Computer Vision (ICCV), Venice, Italy, October 2017.
Guo-Jun Qi, Jiliang Tang, Jingdong Wang and Jiebo Luo, "Mixture Factorized Ornstein-Uhlenbeck Processes for Time-Series Forecasting," ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Nova Scotia, Canada, August 2017.
Yuncheng Li, Yale Song, Jiebo Luo, "Improving Pairwise Ranking for Multi-label Image Classification," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, July 2017.
Yi Wang, Xin Fan, Maomao Min, Jiebo Luo, "Fast Online Incremental Learning on Mixture Streaming Data," The 31st AAAI Conference on Artificial Intelligence (AAAI), San Francisco, CA, February 2017.

2016

Xianchao Zhang, Haixing Li, Wenxin Liang, Jiebo Luo, "Multi-Type Co-clustering of General Heterogeneous Information Networks," IEEE International Conference on Data Mining (ICDM), December 2016.
Young Chol Song, Iftekhar Naim, Abdullah Al Mamun, Kaustubh Kulkarni, Parag Singla, Jiebo Luo, Daniel Gildea and Henry Kautz, "Unsupervised Alignment of Actions in Video with Text Descriptions," AAAI International Joint Conference on Artificial Intelligence (IJCAI), New York City, July 2016.

2015

Iftekhar Naim, Young Chol Song, Henry Kautz, Jiebo Luo, Qiguang Liu, Daniel Gildea, and Liang Huang, "Discriminative Unsupervised Alignment of Natural Language Instructions with Corresponding Video Segments," North American Chapter of the Association for Computational Linguistics (NAACL), 2015.
Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang, "Robust Image Sentiment Analysis using Progressively Trained and Domain Transferred Deep Networks," The Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), Austin, TX, January 25-30, 2015. [PDF] [Project Page]

2010-2014

Iftekhar Naim, Young Song, Daniel Gildea, Qiguang Liu, Henry Kautz and Jiebo Luo, "Unsupervised Alignment of Natural Language Instructions with Video Segments," The Twenty-Eighth AAAI Conference on Artificial Intelligence (AAAI), Quebec City, Canada, July 27-31, 2014.
Yang Cong, Ji Liu, Junsong Yuan, Jiebo Luo, "Self-Supervised Online Metric Learning With Low Rank Constraint for Scene Categorization." IEEE Transactions on Image Processing, 22(8): 3179-3191, 2013.
Liangliang Cao, Xin Jin, Zhijun Yin, Andrey Del Pozo, Jiebo Luo, Jiawei Han, Thomas S. Huang, "RankCompete: Simultaneous ranking and clustering of information networks," Neurocomputing, 95: 98-104, 2012.
Yahong Han, Fei Wu, Qi Tian, Yueting Zhuang, Jiebo Luo, "Correlated Attribute Transfer with Multi-task Graph-Guided Fusion," ACM Multimedia Conference, Nara, Japan, October 2012. (Long paper)
Lixin Duan, Dong Xu, Ivor Tsang, Jiebo Luo, "Visual Event Recognition in Videos by Learning from Web Data," IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, June 2010. (Best Student Paper) (Project Page)

web and social media data mining

2025

Hanjia Lyu, Jinfa Huang, Daoan Zhang, Yongsheng Yu, Xinyi Mou, Jinsheng Pan, Zhengyuan Yang, Zhongyu Wei, Jiebo Luo, "GPT-4V(ision) as A Social Media Analysis Engine," ACM Transactions on Intelligent Systems and Technology, 2025.
Jinfa Huang*, Jinsheng Pan*, Zhongwei Wan, Hanjia Lyu, Jiebo Luo, "Evolver: Chain-of-Evolution Prompting to Boost Large Multimodal Models for Hateful Meme Detection," International Conference on Computational Linguistics (COLING), Abu Dhabi, UAE, 2025.

2024

Zhongyi Qiu, Kangyi Qiu, Hanjia Lyu, Wei Xiong, Jiebo Luo, "Semantics Preserving Emoji Recommendation with Large Language Models," IEEE Big Data Conference, Washington DC, December 2024.
Yiyao Tao*, Hengyu Zhang*, Babli Dey, Selenge Tulga, Hanjia Lyu, Jiebo Luo, "In the Eyes of the Bystander: Are the Stances on Different Conflicts Correlated?" IEEE Big Data Conference, Washington DC, December 2024.
Yulu Qiu, Yutong Ye, Xupin Zhang, and Jiebo Luo, "Moral Frameworks and Sentiment in Tweets: A Comparative Study of Public Opinion on the Israeli-Palestine Conflict," IEEE Big Data Conference, Washington DC, December 2024.
Weihong Qi, Jinsheng Pan, Hanjia Lyu, and Jiebo Luo, "Excitements and Concerns in the Post-ChatGPT Era: Deciphering Public Perception of AI through Social Media Analysis," Telematics and Informatics, 2024.
Xinnong Zhang*, Haoyu Kuang*, Xinyi Mou, Hanjia Lyu, Kun Wu, Siming Chen, Jiebo Luo, Xuanjing Huang, and Zhongyu Wei, "SoMeLVLM: A Large Vision Language Model for Social Media Processing," Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, August 2024.
Xinyi Mou, Zejun Li, Hanjia Lyu, Jiebo Luo, Zhongyu Wei, "Unifying Local and Global Knowledge: Empowering Large Language Models as Political Experts with Knowledge Graphs," The Web Conference (WWW), Singapore, May 2024.
Hanjia Lyu, Weihong Qi, Zhongyu Wei, and Jiebo Luo, "Human vs. LMMs: Exploring the Discrepancy in Emoji Interpretation and Usage in Digital Communication," AAAI International Conference on Web and Social Media (ICWSM), Buffalo, NY, June 2024.
Hanjia Lyu*, Jinsheng Pan*, Zichen Wang*, and Jiebo Luo, "Computational Assessment of Hyperpartisanship in News Titles," AAAI International Conference on Web and Social Media (ICWSM), Buffalo, NY, June 2024.

2023

Jinsheng Pan, Zichen Wang, Weihong Qi, Hanjia Lyu, and Jiebo Luo, "Understanding Divergent Framing of the Supreme Court Controversies: Social Media vs. News Outlets," IEEE International Conference on Big Data, Sorrento, Italy, December 2023.
Michael Reidy, Henry Mallon, and Jiebo Luo, "Investigating the Effectiveness of Deep Learning and CFA Interpolation Based Classifiers on Identifying AIGC," IEEE International Conference on Big Data, Sorrento, Italy, December 2023.
Enting Zhou*, Yurong Liu*, Hanjia Lyu, Jiebo Luo, "A Fine-Grained Analysis of Public Opinion toward Chinese Technology Companies on Reddit," IEEE International Conference on Big Data, Sorrento, Italy, December 2023.
Edinam Klutse, Samuel Nuamah-Amoabeng, Hanjia Lyu and Jiebo Luo, "Dismantling Hate: Understanding Hate Speech Trends Against NBA Athletes," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Hybrid, Pittsburgh, PA, September 2023.
Jinsheng Pan, Weihong Qi, Zichen Wang, Hanjia Lyu and Jiebo Luo, "Bias or Diversity? Unraveling Semantic Discrepancy in U.S. News Headlines," Mediate Workshop, AAAI International Conference on Web and Social Media (ICWSM), Limassol, Cyprus, June 5, 2023.
Hanjia Lyu, Arsal Imtiaz, Yufei Zhao, Jiebo Luo, "Human Behavior in the Time of COVID-19: Learning from Big Data," Frontiers in Big Data, 2023.

2022

Sajid Hussain Rafi Ahamed, Shahid Shakil, Hanjia Lyu, Xinping Zhang, and Jiebo Luo, "Doctors vs. Nurses: Understanding the Great Divide in Vaccine Hesitancy among Healthcare Workers," IEEE International Conference on Big Data, Osaka, Japan, December 2022.
Yufei Zhao, Chao Huang, and Jiebo Luo, "How to Prepare for the Next Pandemic - Investigation of Correlation Between Food Prices and COVID-19 From Global and Local Perspectives," IEEE International Conference on Big Data, Osaka, Japan, December 2022.
Yangxin Fan, Hanjia Lyu, Jin Xiao, and Jiebo Luo, "American Twitter Users Revealed Social Determinants-related Oral Health Disparities amid the COVID-19 Pandemic," Quintessence International, 2022.
Junda Wang, Xupin Zhang, Jiebo Luo, "How COVID-19 Have Changed Crowdfunding: Evidence from GoFundMe," Frontiers on Computer Science, section Human-Media Interaction, Vol. 4, 2022.
Arsal Imtiaz, Danish Khan, Hanjia Lyu, and Jiebo Luo, "Taking sides: Public Opinion over the Israel-Palestine Conflict in 2021," International Workshop on Social Sensing (SocialSens): Special Edition on Belief Dynamics, AAAI International Conference on Web and Social Media (ICWSM), Atlanta, GA, June 2022.
Yichi Qian, Qiyi Shan, Hanjia Lyu, and Jiebo Luo, "Look behind the Censorship: Reposting-User Characterization and Muted-Topic Restoration," International Workshop on Social Sensing (SocialSens): Special Edition on Belief Dynamics, AAAI International Conference on Web and Social Media (ICWSM), Atlanta, GA, June 2022.
Yingqiang Ge, Juntao Tan, Yan Zhu, Yinglong Xia, Jiebo Luo, Shuchang Liu, Zuohui Fu, Shijie Geng, Zelong Li and Yongfeng Zhang, "Explainable Fairness for Feature-aware Recommender Systems," ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2022), July 11 - 15, 2022, Madrid, Spain.
Hanjia Lyu, Zihe Zheng, and Jiebo Luo, "Misinformation versus Facts: Understanding the Influence of News Regarding COVID-19 Vaccines on Vaccine Uptake," Health Data Science, 2022.
Jian Kang, Yan Zhu, Yinglong Xia, Jiebo Luo, Hanghang Tong. "RawlsGCN: Towards Rawlsian Difference Principle on Graph Convolutional Network," ACM Web Conference (WWW), 2022.

2021

Hanjia Lyu, Yangxin Fan, Ziyu Xiong, Mayya Komisarchik, and Jiebo Luo, "Understanding Public Opinion toward the #StopAsianHate Movement and the Relation with Racially Motivated Hate Crimes in the US," IEEE Transactions on Computational Social Systems, in press.
Haitian Zheng, Kefei Wu, Jong-Hwi Park, Wei Zhu, and Jiebo Luo, "Personalized Fashion Recommendation from Personal Social Media Data: An Item-to-Set Metric Learning Approach," IEEE International Conference on Big Data, December 2021.
Tanqiu Jiang, Sidhant Bendre, Hanjia Lyu, and Jiebo Luo, "From Static to Dynamic Prediction: Wildfire Risk Assessment Based on Multiple Environmental Factors," Special Session on Intelligent Data Mining, IEEE International Conference on Big Data, December 2021.
Xupin Zhang, Hanjia Lyu, and Jiebo Luo, "Understanding the Hoarding Behaviors during the COVID-19 Pandemic using Large Scale Social Media Data," Special Session on Intelligent Data Mining, IEEE International Conference on Big Data, December 2021.
Songyang Zhang, Tolga Aktas, and Jiebo Luo, "Mi YouTube es Su YouTube? Analyzing the Cultures using YouTube Thumbnails of Popular Videos," Special Session on Intelligent Data Mining, IEEE International Conference on Big Data, December 2021.
Hanjia Lyu, Junda Wang, Wei Wu, Viet Duong, Xiyang Zhang, Timothy D. Dye, and Jiebo Luo, "Social Media Study of Public Opinions on Potential COVID-19 Vaccines: Informing Dissent, Disparities, and Dissemination," Intelligent Medicine, 2021.
Junda Wang, Xupin Zhang, Jiebo Luo, "How COVID-19 Has Changed the World of Crowdfunding," Sustainability, 2021.
Ziyu Xiong, Pin Li, Hanjia Lyu, and Jiebo Luo, "Social Media Opinions on Working From Home in the United States During the COVID-19 Pandemic: Observational Study," JMIR Medical Informatics, 2021.
Wei Wu, Hanjia Lyu, Jiebo Luo, "Characterizing Discourse about COVID-19 Vaccines: A Reddit Version of the Pandemic Story," Health Data Science, 2021.
Xupin Zhang, Hanjia Lyu, Jiebo Luo, "What Contributes to a Crowdfunding Campaign's Success?
Evidence and Analyses from GoFundMe Data," IEEE Journal of Social Computing, 2021.
Zhongkai Shangguan, Zihe Zheng and Jiebo Luo, "What Makes A Turing Award Winner?" International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Virtual, July 2021.
Long Chen, Hanjia Lyu, Tongyu Yang, Yu Wang and Jiebo Luo, "Fine-Grained Analysis of the Use of Neutral and Controversial Terms for COVID-19 on Social Media," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Virtual, July 2021.
Karan Vombatkere, Hanjia Lyu and Jiebo Luo, "How Political is the Spread of COVID-19 in the United States? An Analysis using Transportation and Weather Data," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Virtual, July 2021.
Xupin Zhang, Hanjia Lyu and Jiebo Luo, "Understanding the Hoarding Behaviors duringthe COVID-19 Pandemic using Large Scale Social Media Data," Working Paper, International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Virtual, July 2021.
Ziqi Tang, Yutong Wang and Jiebo Luo, "Are Top School Students More Critical of Their Professors? Mining Comments on RateMyProfessors.com," Working Paper, International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Virtual, July 2021.
Xiyang Zhang, Yu Wang, Hanjia Lyu, Yipeng Zhang, Yubao Liu, Jiebo Luo, "The Influence of COVID-19 on people's Well-Being: Big Data Methods for Capturing Working Adults' Well-being and Protective Factors Nationwide," Frontiers in Psychology, 2021.
Yipeng Zhang, Hanjia Lyu. Yubao Liu. Xiyang Zhang, Yu Wang, Jiebo Luo, "Monitoring Depression Trend on Twitter during the COVID-19 Pandemic: Observational Study," Journal of Medical Internet Research Infodemiology, 2021.

2020

Li Sun*, Haoqi Zhang*, Songyang Zhang, and Jiebo Luo, "Content-based Analysis of the Cultural Differences between TikTok and Douyin," Special Session on Intelligent Data Mining, IEEE International Conference on Big Data, Atlanta, GA, December 2020.
Neil Yeung, Jonathan Lai, and Jiebo Luo, "Face Off: Polarized Public Opinions on Personal Face Mask Usage during the COVID-19 Pandemic," Special Session on Intelligent Data Mining, IEEE International Conference on Big Data, Atlanta, GA, December 2020.
Wei Wu*, Jinlong Ruan*, and Jiebo Luo, "Stock Price Prediction Under Anomalous Circumstances," Special Session on Intelligent Data Mining, IEEE International Conference on Big Data, Atlanta, GA, December 2020.
Shuaidong Pan*, Faner Lin*, and Jiebo Luo, "Do Sports and Politics Mix? Cross-Analysis of Fan Bases of Major League Sports and Presidential Candidates," IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2020.
Viet Duong, Phu Pham, Tongyu Yang, Yu Wang and Jiebo Luo, "The Ivory Tower Lost: How College Students Respond Differently than the General Public to the COVID-19 Pandemic," IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2020.
Luoying Yang, Zhou Xu, Jiebo Luo, "Measuring Female Representation and Impact in Films over Time," ACM Transactions on Data Science, 2020.
Hanjia Lyu, Long Chen, Yu Wang, Jiebo Luo, "Sense and Sensibility: Characterizing Social Media Users Regarding the Use of Controversial Terms for COVID-19," IEEE Transactions on Big Data, 2020.
Xinyi Lu, Long Chen, Jianbo Yuan, Joyce Luo, Zidian Xie, Jiebo Luo, Dongmei Li, "E-cigarette Flavors and Their Perceptions on Social Media: Observational Study," Journal of Medical Internet Research, Vol 22, No 6, 2020.

2019

Shuaidong Pan, Tianran Hu, Shujing Sun, Jianbo Yuan, and Jiebo Luo, "Help Oneself in Helping the Others: The Ecology of Online Support Groups," IEEE International Conference on Big Data, Los Angeles, CA, December 2019.
Yingtong Dou, Weijian Li, Zhirong Liu, Zhenhua Dong, Jiebo Luo and Philip Yu, "Uncovering Download Fraud Activities in Mobile App Market," IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), Vancouver, Canada, August 2019.
Yuan Liu, Zhongwei Cheng, Jie Liu, Bourhan Yassin, Zhe Nan, Jiebo Luo, "AI for Earth: Rainforest Conservation by Acoustic Surveillance," Workshop on Data Mining and AI for Conservation, ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Anchorage, AK, August 2019.
Tianran Hu, Yinglong Xia, Jiebo Luo, "To Return or to Explore: Modelling Human Mobility and Dynamics in Cyberspace," The Web Conference (formerly WWW), San Francisco, CA, May 2019.

2018

Zhongping Zhang, Tianlang Chen, Zheng Zhou, Jiaxin Li, and Jiebo Luo, "How to Become Instagram Famous: Post Popularity Prediction with Dual-Attention," IEEE International Conference on Big Data, Seattle, WA, December 2018.
Tianlang Chen, Yuxiao Chen, Han Guo, and Jiebo Luo, "You Type a Few Words and We Do the Rest: Image Recommendation for Social Multimedia Posts," IEEE International Conference on Big Data, Seattle, WA, 2018.
Benjamin Kane and Jiebo Luo, "Do the Communities We Choose Shape our Political Beliefs? A Study of the Politicization of Topics in Online Social Groups," International Workshop on Big Social Media Data Management and Analysis, IEEE International Conference on Big Data, Seattle, WA, December 2018.
Yuxiao Chen*, Jianbo Yuan*, Quanzeng You, Jiebo Luo, "Twitter Sentiment Analysis via Bi-sense Emoji Embedding and Attention-based LSTM," ACM Multimedia Conference, Seoul, Korea, October 2018.
Xuefeng Peng, Li-Kai Chi and Jiebo Luo, "The Effect of Pets on Happiness: A Large-scale Multi-Factor Analysis using Social Multimedia," ACM Transactions on Intelligent Systems and Technology (TIST) 9(5), 2018.
Tianran Hu, Jiebo Luo and Wei Liu, "Life in the 'Matrix': Human Mobility Patterns in the Cyber Space," AAAI International Conference on Web and Social Media (ICWSM), Stanford, CA, June 2018. (Full Paper Acceptance Rate 16%)
Weijian Li*, Yuxiao Chen*, Tianran Hu, and Jiebo Luo, "Mining the Relationship between Emoji Usage Patterns and Personality," AAAI International Conference on Web and Social Media (ICWSM), Stanford, CA, June 2018.
Tianlang Chen, Yuxiao Chen, Han Guo and Jiebo Luo, "When E-commerce Meets Social Media: Identifying Business on WeChat Moment Using Bilateral-Attention LSTM," The Web Conference (WWW), Lyon, France, April 2018.
Peijun Zhao, Jia Jia, Lexing Xie, Yongsheng An and Jiebo Luo, "Analyzing and Predicting Emoji Usages in Social Media," The Web Conference (WWW), Lyon, France, April 2018.

2017

Yiming Pan, Xuefeng Peng, Tianran Hu, and Jiebo Luo, "Understanding What affects Career Progression Using LinkedIn and Twitter Data," Special Session on Intelligent Data Mining, IEEE Big Data Conference, Boston, MA, December 2017.
Xuefeng Peng, Yiming Pan, and Jiebo Luo, "Predicting High Taxi Demand Regions Using Social Media Check-ins," Special Session on Intelligent Data Mining, IEEE Big Data Conference, Boston, MA, December 2017.
Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, Yu Wang and Jiebo Luo, "Multimodal Fusion with Recurrent Neural Networks for Rumor Detection on Microblogs," ACM Multimedia Conference, Mountain View, CA, October 2017.
Yu Wang, Jiebo Luo and Xiyang Zhang, "When Follow is Just One Click Away: Understanding Twitter Follow Behavior in the 2016 U.S. Presidential Election," International Conference on Social Informatics (SocInfo), Oxford, England, Sept. 2017.
Yu Wang, Yang Feng and Jiebo Luo, "How Polarized Have We Become? A Multimodal Classification of Trump Followers and Clinton Followers," International Conference on Social Informatics (SocInfo), Oxford, England, Sept. 2017.
Zhiwei Jin, Juan Cao, Han Guo, Yongdong Zhang, Yu Wang and Jiebo Luo, "Detection and Analysis of 2016 US Presidential Election Related Rumors on Twitter," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Washington, DC, July 2017.
Yu Wang and Jiebo Luo, "Inferring Follower Preferences with Sparse Learning," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Washington, DC, July 2017.
Yu Wang and Jiebo Luo, "Gender Politics in the 2016 Presidential Election: A Computer Vision Approach," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), July 2017.
Quanzeng You, Jungseock Joo, Dario Garcia Garcia, Jiebo Luo, "Cultural Dynamics and Trends in Facebook Photographs," AAAI International Conference on Web and Social Media (ICWSM), Montreal, Canada, May 2017.
Tianran Hu, Han Guo, Hao Sun, Thuy-vy Thi Nguyen, Jiebo Luo, "Spice up Your Chat: The Intentions and Sentiment Effects of Using Emojis," AAAI International Conference on Web and Social Media (ICWSM), Montreal, Canada, May 2017.
Tianran Hu, Ruihua Song, Xing Xie, Maya Abtahian, Philip Ding and Jiebo Luo, "A World of Difference: Divergent Word Interpretations among People," AAAI International Conference on Web and Social Media (ICWSM), Montreal, Canada, May 2017.
Rijul Magu, Kshitij Joshi and Jiebo Luo, "Decoding the Hate Code on Social Media," AAAI International Conference on Web and Social Media (ICWSM), Montreal, Canada, May 2017.
Yu Wang and Jiebo Luo, "Tactics and Tallies: A Study of the 2016 U.S. Presidential Campaign Using Twitter 'Likes'," AAAI International Conference on Web and Social Media (ICWSM), Second International Workshop on News and Public Opinion, Montreal, Canada, May 2017.
Kuan-Ting Chen and Jiebo Luo, "When Fashion Meets Big Data: Discriminative Mining of Best Selling Clothing Features," World Wide Web Conference (WWW), Perth, Australia, April 2017.
Tianlang Chen, Yuxiao Chen and Jiebo Luo, "A Selfie is Worth a Thousand Words: Mining Personal Patterns behind User Selfie-posting Behaviours," World Wide Web Conference (WWW), Perth, Australia, April 2017.
Tianran Hu, Eric Biglow, Henry Kautz, and Jiebo Luo, "Tales of Two Cities: Using Social Media to Understand Idiosyncratic Lifestyles in Distinctive Metropolitan Areas," Special Issue on Big Media Data: Understanding, Search, and Mining, IEEE Transactions on Big Data, 3(1): 55-66, 2017.

2016

Yiheng Zhou, Numair Sani, and Jiebo Luo, "Fine-grained Mining of Illicit Drug Use Patterns Using Social Multimedia Data from Instagram," Special Session on Intelligent Data Mining," IEEE International Conference on Big Data (Big Data), Washington, DC, December 2016.
Jianbo Yuan, Walid Shalaby, Mohammed Korayem, David Lin, Khalifeh AlJadda and Jiebo Luo, "Solving Cold Start Problem in Large-scale Recommendation Engines: A Deep Learning Approach," IEEE International Conference on Big Data (Big Data), 2016.
Yuchen Wu, Jianbo Yuan, Quanzeng You, and Jiebo Luo, "The Effect of Pets on Happiness: A Data-Driven Approach via Large-Scale Social Media," Special Session on Intelligent Data Mining, IEEE International Conference on Big Data (Big Data), Washington, DC, December 2016.
Haofu Liao, Yuncheng Li, Tianran Hu, and Jiebo Luo, "Inferring Restaurant Styles by Mining Crowd Sourced Photos from User-Review Websites," IEEE International Conference on Big Data (Big Data), Washington, DC, December 2016.
Yu Wang, Yang Feng, Jiebo Luo, "Pricing the Woman Card: Gender Politics in 2016 US Presidential Election", Workshop on Applications of Big Data in Computational Social Science, IEEE International Conference on Big Data (Big Data), Washington, DC, December 2016.
Yu Wang, Yang Feng, Xiyang Zhang, and Jiebo Luo, "Inferring Voter Preferences behind Brexit," Workshop on Application of Big Data for Computational Social Science, IEEE International Conference on Big Data (Big Data), Washington, DC, December 2016.
Yang Feng, Jiebo Luo, "When Do Luxury Cars Hit Road? Findings by A Big Data Approach", Workshop on Applications of Big Data in Computational Social Science, IEEE International Conference on Big Data (Big Data), Washington, DC, December 2016.
Yu Wang, Yang Feng, Jiebo Luo, Xiyang Zhang, "Voting with Feet: Who are Leaving Hillary Clinton and Donald Trump?" IEEE International Symposium on Multimedia, San Jose, CA, December 2016. (Invited Paper)
Tianran Hu, Ruihua Song, Xing Xie, Jiebo Luo, "Mining Shopping Patterns for Divergent Urban Regions by Incorporating Mobility Data", ACM International Conference on Information and Knowledge Management (CIKM), Indianapolis, IN, October 2016.
Kelly He, Lee Murphy, and Jiebo Luo, "Using Social Media to Promote STEM Education: Matching College Students with Role Models", European Conference on Machine Learning and Principles and Practice of Knowledge Discovery (ECML/PKDD), Riva del Garda, Italy, September 2016. [arXiv]
Zhishen Pan, Kevin Chi, Timothy Dye, Jiebo Luo, "Towards Understanding How News Coverage Affect Public Perception During Epidemic Outbreaks", International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Washington DC, June 2016.
Yiheng Zhou, Numair Sani, Jiebo Luo, "Understanding Illicit Drug Use Behaviors by Mining Instagram," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Washington DC, June 2016. [arXiv]
Tianran Hu, Haoyuan Xiao, Jiebo Luo, Thuy-vy Thi Nguyen, "What the Language You Tweet Says About Your Occupation," AAAI International Conference on Weblogs and Social Media (ICWSM), Cologne, Germany, May 2016.
Yu Wang, Yuncheng Li, Jiebo Luo, "Deciphering the 2016 U.S. Presidential Campaign in the Twitter Sphere: A Comparison of the Trumpists and Clintonists," AAAI International Conference on Weblogs and Social Media (ICWSM), Cologne, Germany, May 2016. [arXiv]
Yu Wang, Jiebo Luo, Richard G. Niemi, Yuncheng Li, Tianran Hu, "Catching Fire via 'Likes': Inferring Topic Preferences of Trump Followers on Twitter," AAAI International Conference on Weblogs and Social Media (ICWSM), Cologne, Germany, May 2016. [arXiv]
Nabil Hossain, Tianran Hu, Roghayeh Feizi, Ann Marie White, Jiebo Luo, Henry Kautz, "Inferring Fine-grained Details on User Activities and Home Location from Social Media: Detecting Drinking-While-Tweeting Patterns in Communities," AAAI International Conference on Weblogs and Social Media (ICWSM), Cologne, Germany, May 2016. [arXiv]
Yu Wang, Yuncheng Li, Richard G. Niemi, Jiebo Luo, "To Follow or Not to Follow: Analyzing the Growth Patterns of the Trumpists on Twitter," AAAI International Conference on Weblogs and Social Media (ICWSM), Workshop on Social Media in the Newsroom, Cologne, Germany, May 2016.
Zhiwei Jin, Juan Cao, Yongdong Zhang, Jiebo Luo, "News Verification by Exploiting Conflicting Social Viewpoints in Microblogs," The 30th AAAI Conference on Artificial Intelligence (AAAI), Phoenix, AZ, January 2016. [PDF]

2015

Quanzeng You, Sumit Bhatia, Jiebo Luo, "A Picture Tells a Thousand Words About You! User Interest Profiling from User Generated Visual Content," Signal Processing, Special Issues on Big Data Meets Multimedia Analytics, December 2015. [PDF]
Quanzeng You, Liangliang Cao, Jiebo Luo, "A Multifaceted Social Multimedia-based Approach to Prediction of Elections,", IEEE Transactions on Multimedia, to appear, 2015. [PrePrintPDF]
Tianran Hu, Adam Sadilek, Henry Kautz, and Jiebo Luo, "Home Location Inference from Sparse and Noisy Data: Models and Applications," IEEE International Conference on Data Mining (ICDM), Workshop on Social Multimedia Data Mining, Atlantic City, December 2015.
Yu Wang, Jianbo Yuan, and Jiebo Luo, "To Love or to Loathe: How is the World Reacting to China's Rise?" IEEE International Conference on Data Mining (ICDM), Workshop on Big Media Data: Understanding, Search, and Mining, Atlantic City, December 2015. [PrePrintPDF]
Ran Pang, Agustin Baretto, Henry Kautz, and Jiebo Luo, "Monitoring Adolescent Alcohol Use via Multimodal Data Analysis in Social Multimedia," Special Session on Intelligent Mining, IEEE Big Data Conference, Santa Clara, CA, October 2015. [PrePrintPDF]
Yuncheng Li, Yang Cong, Tao Mei, and Jiebo Luo, "User-Curated Image Collections: Modeling and Recommendation," IEEE Big Data Conference, Santa Clara, CA, October 2015. [PrePrintPDF]
Yu Wang, Jianbo Yuan, and Jiebo Luo, "America Tweets China: A Fine-Grained Analysis of the State and Individual Characteristics Regarding Attitudes towards China," IEEE Big Data Conference, Santa Clara, CA, October 2015. [PrePrintPDF]
Kuan-Ting Chen*, Kezhen Chen*, Peizhong Cong, Winston Hsu, Jiebo Luo, "Who are the Devils Wearing Prada in New York City?" ACM Multimedia Conference, October 2015. [PrePrintPDF]
Xitong Yang, Yuncheng Li, Jiebo Luo, "Pinterest Board Recommendation for Twitter Users," ACM Multimedia Conference, October 2015. [PrePrintPDF]
Danning Zheng, Tianran Hu, Quanzeng You, and Jiebo Luo, "Towards Lifestyle Understanding: Predicting Home and Vacation Locations from User's Online Photo Collections," AAAI International Conference on Weblogs and Social Media (ICWSM), May 2015.

2010-2014

Quanzeng You, Sumit Bhatia, Jiebo Luo, "The eyes of the beholder: Gender prediction using images posted in online social networks," IEEE International Conference on Data Mining (ICDM), Workshop on Social Multimedia Data Mining, December 2014. [PDF]
Simon Weber, Jiebo Luo, "What Makes An Open Code Popular in Github," IEEE International Conference on Data Mining (ICDM), Workshop on Software Data Mining, December 2014.
Andrew Nocka, Danning Zheng, Tianran Hu, Jiebo Luo, "Moneyball for Academia: Towards Measuring and Maximizing Faculty Performance and Impact," IEEE International Conference on Data Mining (ICDM), Workshop on Domain Dependent Data Mining, December 2014.
Danning Zheng, Tianran Hu, Quanzeng You, and Jiebo Luo, "Inferring Home Location from User's Photo Collections based on Visual Content and Mobility Patterns," ACM Multimedia Conference, Workshop on Geotagging in Multimedia (GeoMM), November 2014.
Yuncheng Li, Jiebo Luo, Tao Mei, "Personalized Image Recommendation for Web Search Engine Users", IEEE ICME, July 2014.
Junhuan Zhu, Quanzeng You, Jiebo Luo and John R. Smith, "Towards Understanding the Effectiveness of Election Related Images in Social Media," IEEE International Conference on Data Mining (ICDM), Workshop on Domain-driven Data Mining, December 2013. [PDF]
Jianbo Yuan, Quanzeng You, and Jiebo Luo, "Are There Cultural Differences in Event Driven Information Propagation Over Social Media?" ACM Multimedia Conference, International Workshop on Socially-Aware Multimedia (IWSAM), October 2013. [PDF]
Quanzeng You and Jiebo Luo, "Towards Social Imagematics: Sentiment Analysis in Social Multimedia," ACM SIGKDD, Workshop on Multimedia Data Mining, August 2013. [PDF]
Jianbo Yuan, Quanzeng You, Sean McDonough, and Jiebo Luo, "Sentribute: Image Sentiment Analysis from a Mid-level Perspective," ACM SIGKDD, Workshop on Issues of Sentiment Discovery and Opinion Mining (WISDOM), August 2013. [PDF]
Ge Ma and Jiebo Luo, "Is A Social Picture Worth 1000 Votes? Analyzing the Sentiment of Election Related Photos," IEEE ICME, July 2013.
Xin Jin, Jie Yu, Jiawei Han, Jiebo Luo, "Reinforced retrieval in image-rich information networks via integration of link and content based similarities," IEEE Transactions on Knowledge Discovery and Engineering, 25(2): pp 448-460, Feb. 2013.
Xin Jin, Cindy Lin, Jiebo Luo, Jiawei Han, "SocialSpamGuard: A Data Mining-based Spam Detection System for Social Media Networks," Demo Paper, International Conference on Very Large Data Bases (VLDB), Seattle, WA, August 2011.
Xin Jin, Chi Wang, Jiebo Luo, Jiawei Han, "LikeMiner: A System for Mining the Power of 'Like' in Social Media Networks," Demo Paper, International Conference on Knowledge Discovery and Data Mining (KDD), San Diego, CA, August 2011.
Zhijun Yin, Liangliang Cao, Jiawei Han, Jiebo Luo, and Thomas Huang, "Diversified Trajectory Pattern Ranking in Geo-tagged Social Media," SIAM Conference on Data Mining (SDM), Mesa, AZ, April 2011.
Jie Yu, Xin Jin, Jiawei Han, Jiebo Luo, "Collection-based sparse label propagation and its application to social group suggestion from photos," ACM Transactions on Intelligent Systems and Technology, 2(2): , February 2011.
Xin Jin, Andrew Gallagher, Jiawei Han, Jiebo Luo, "Wisdom of Social Multimedia: Using Flickr for Prediction and Forecast," ACM Multimedia Conference, Brave and New Ideas Track, Firenze, Italy, October 2010. (Long Paper)

biomedical analytics and health informatics

2025

Marios Kritsilis, Lotte Vanherle, Marko Rosenholm, Rene In 't Zandt, Yuan Yao, Kelley M. Swanberg, Pia Weikop, Michael Gottschalk, Nagesh C. Shanbhag, Jiebo Luo, Kimberly Boster, Maiken Nedergaard, Anja Meissner, Iben Lundgaard, "Loss of glymphatic homeostasis in heart failure," Brain, 2025.
Fenglin Liu, Zheng Li, Qingyu Yin, Jinfa Huang, Xian Wu, Anshul Thakur, Kim Branson, Patrick Schwab, Bing Yin, Yefeng Zheng, Jiebo Luo, and David Clifton, "A Multimodal Multidomain Multilingual Medical Foundation Model for Zero Shot Clinical Diagnosis", npj Digital Medicine, 2025.

2024

Ziyun Zeng, Ashwin Ramesh, Jinglong Ruan, Peirong Hao, Nisreen Al Jallad, Hoonji Jang, Oriana Ly-Mapes, Kevin Fiscella, Jin Xiao, Jiebo Luo, "Use Artificial Intelligence to Detect Dental Caries on Intraoral Photos," Quintessence International, 2024.
Junda Wang, Weijian Li, Han Wang, Hanjia Lyu, Caroline Thirukumaran, Addisu Mesfin, Hong Yu, Jiebo Luo, "CRTRE: Causal Rule Generation with Target Trial Emulation Framework," IEEE Big Data Conference, Washington DC, December 2024.
Wenting Chen, Linlin Shen, Jingyang Lin, Jiebo Luo, Xiang Li, Yixuan Yuan, "Fine-Grained Image-Text Alignment in Medical Imaging Enables Explainable Cyclic Image-Report Generation," Annual Meeting of the Association for Computational Linguistics (ACL), Bangkok, Thailand, August 2024.
Jixuan Leng, Junfei Liu, Galen Cheng, Haohan Wang, Scott Quarrier, Jiebo Luo, Rajat Jain, "UroSAM: A Machine Learning Model to Automatically Identify Kidney Stone Composition from Endoscopic Video," Journal of Endourology, Special Issue on Artificial Intelligence in Endourology, Stone Disease, and BPH, 2024.
Wenxi Yue, Jing Zhang, Kun Hu, Yong Xia, Jiebo Luo, Zhiyong Wang, "SurgicalSAM: Efficient Class Promptable Surgical Instrument Segmentation," The 38th AAAI Conference on Artificial Intelligence (AAAI), February 2024.

2023

Zhihong Zhang, Qinqin Xiao, Jiebo Luo, "Infant death prediction using machine learning: A population-based retrospective study," Computers in Biology and Medicine 165:107423, October 2023.
Jingyuan Chen, Yuan Yao, Mie Anderson, Natalie Hauglund, Celia Kjaerby, Verena Untiet, Maiken Nedergaard and Jiebo Luo, "Self-distilled Hierarchical Mixture of Modality Experts Transformer are Efficient Mice Sleep Learners," IEEE International Conference on Digital Health, Chicago, IL, July 2023. (Best Student Paper)
Weijian Li, Wei Zhu, Ray Dorsey and Jiebo Luo, "Remote Medication Status Prediction for Individuals with Parkinson's Disease using Time-series Data from Smartphones," IEEE International Conference on Digital Health, Chicago, IL, July 2023.
Kevin Kapcio, Hanjia Lyu, Kyle Purrman, Christian Peyre, Carolyn Jones, Jiebo Luo and Michal Lada, "Applying Machine Learning to Predict Esophageal Cancer Recurrence after Esophagectomy," IEEE International Conference on Digital Health, Chicago, IL, July 2023.
Jingyang Lin, Junyu Chen, Hanjia Lyu, Igor Khodak, Divya Chhabra, Colby L Day Richardson, Irina Prelipcean, Andrew M Dylag, Jiebo Luo, "Predicting Adverse Neonatal Outcomes for Preterm Neonates with Multi-Task Learning," IEEE International Conference on Digital Health, Chicago, IL, July 2023.
Wenxi Yue, Hongen Liao, Yong Xia, Vincent Lam, Jiebo Luo, Zhiyong Wang, "Cascade Multi-Level Transformer Network for Surgical Workflow Analysis," IEEE Transactions on Medical Imaging, Early Access, 2023.
Wei Zhu, Runtao Zhou, Yuan Yao, Timothy Campbell, Rajat Kumar Jain, and Jiebo Luo, "SegPrompt: Using Segmentation Map as A Better Prompt to Finetune Deep Models for Kidney Stone Classification," Medical Imaging with Deep Learning (MIDL), 2023.
Shubing Cai, Weijian Li, Orna Intrator, Rajesh Makineni, Peter Veazie, Jiebo Luo, "Race and COVID Severity among Veterans: Applications of Machine Learning," VA Health Services Research & Development National Meeting, 2023.

2022

Zidian Xie, Jinlong Ruan, Yifan Jiang, Bokai Zhang, Tianlang Chen, Jiebo Luo, Dongmei Li, "Potential Impact of FDA Flavor Enforcement Policy on Vaping Behavior on Twitter," International Journal of Environmental Research and Public Health, 2022.
Yangxin Fan, Hanjia Lyu, Jin Xiao, and Jiebo Luo, "American Twitter Users Revealed Social Determinants-related Oral Health Disparities amid the COVID-19 Pandemic," Quintessence International, 2022.
Peter Joo, Weijian Li, Amy Phan, Gabriel Ramirez, Caroline P Thirukumaran, Jiebo Luo, Emmanuel N Menga, Addisu Mesfin, "Health care disparities in complication and mortality rates following surgical management of cauda equina syndrome," Spine 22 (9), S52-S53, 2022.
A Kolokythas, W Zhu, J Luo, "Use of Artificial Intelligence in the Characterization of Oral Mucosal Lesions," Journal of Oral and Maxillofacial Surgery 80 (9), S3-S4, 2022.
Timothy Dye, Lisette Alcantara, Hanjia Lyu, Shazia Siddiqi, Saloni Sharma, Eva Pressman, and Jiebo Luo, "Oppression, COVID Vaccination, and Vaccine Sentiments in a Global Sample," Annals of Epidemiology (ACE Abstracts), 2022.
Kevin C. Kapcio, Hanjia Lyu, Kyle C. Purrman, Christian G. Peyre, Jiebo Luo, Carolyn E. Jones, and Michal J. Lada, "Applying Machine Learning To Predict Esophageal Cancer Recurrence After Esophagectomy," Supplemental Issue of Journal of the American College of Surgeons, ACS Clinical Congress, October 2022.
Wei Zhu, Jiebo Luo, "Federated Medical Image Analysis with Virtual Sample Synthesis," International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Singapore, September 2022.
Wei Zhu, Andrew White, and Jiebo Luo, "Federated Learning of Molecular Properties in a Heterogeneous Setting," Cell Patterns, 2022.
Nisreen Al-Jallad, Oriana Ly-Mapes, Peirong Hao, Jinlong Ruan, Ashwin Ramesh, Jiebo Luo, Tong Tong Wu, Timothy Dye, Noha Rashwan, Johana Ren, Hoonji Jang, Luis Mendez, Nora Alomier, Sherita Bullock, Kevin Fiscella, Jin Xiao, "Artificial intelligence-powered smartphone application, AICaries, improves at-home dental caries screening in children: moderated and unmoderated usability tests," Plos Digital Health, 2022.
Wei Xiong, Neil Yeung, Shubo Wang, Liyun Wang, Jiebo Luo, "Breast Cancer Induced Bone Osteolysis Prediction Using Temporal Variational Auto-Encoders," Special Issue on AI for Advanced Biomedical Applications, BME Frontiers, 2022.
Hanjia Lyu, Zihe Zheng, and Jiebo Luo, "Misinformation versus Facts: Understanding the Influence of News Regarding COVID-19 Vaccines on Vaccine Uptake," Health Data Science, 2022.

2021

Jin Xiao, Jiebo Luo, Oriana Ly-Mapes, Tong Tong Wu, Timothy Dye, Nisreen Al Jallad, Peirong Hao, Jinlong Ruan, Sherita Bullock, Kevin Fiscella, "Assess A Smartphone App (AICaries) that Uses Artificial Intelligence to Detect Dental Caries in Children and Provide Interactive Oral Health Education: Protocol for Design and Usability Testing," JMIR Research Protocols, 2021.
Hanjia Lyu, Junda Wang, Wei Wu, Viet Duong, Xiyang Zhang, Timothy D. Dye, and Jiebo Luo, "Social Media Study of Public Opinions on Potential COVID-19 Vaccines: Informing Dissent, Disparities, and Dissemination," Intelligent Medicine, 2021.
Wei Wu, Hanjia Lyu, Jiebo Luo, "Characterizing Discourse about COVID-19 Vaccines: A Reddit Version of the Pandemic Story," Health Data Science, 2021.
Xiyang Zhang, Yu Wang, Hanjia Lyu, Yipeng Zhang, Yubao Liu, Jiebo Luo, "The Influence of COVID-19 on people's Well-Being: Big Data Methods for Capturing Working Adults' Well-being and Protective Factors Nationwide," Frontiers in Psychology, 2021.
Yipeng Zhang, Hanjia Lyu. Yubao Liu. Xiyang Zhang, Yu Wang, Jiebo Luo, "Monitoring Depression Trend on Twitter during the COVID-19 Pandemic: Observational Study," Journal of Medical Internet Research Infodemiology, 2021.
Cheng Peng, Haofu Liao, Gina Wong, Jiebo Luo, S. Kevin Zhou, Rama Chellappa, "XraySyn: Realistic View Synthesis from a Single Radiograph Through CT Prior," The 35th AAAI Conference on Artificial Intelligence (AAAI), February 2021.

2020

Weijian Li, Wei Zhu, Ray Dorsey, Jiebo Luo, "Predicting Parkinson's Disease with Multimodal Irregularly Collected Longitudinal Smartphone Data," IEEE International Conference on Data Mining (ICDM), November 2020.
Zhou Zhuang, Johnson V. John, Haofu Liao, Jiebo Luo, Paul Rubery, Addisu Mesfin, Sunil Kumar Boda, Jingwei Xie and Xinping Zhang, ''Electrospray-Enabled Peptide Coating of Structural Bone Allografts for Enhanced Repair and Reconstruction of Femoral Segmental Defects,'' ACS Biomaterials Science & Engineering, in press.
Wei Zhu, Haofu Liao, Wenbin Li, Weijian Li, Jiebo Luo, "Alleviating the Incompatibility between Cross Entropy Loss and Episode Training for Few-shot Skin Disease Classification," International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Lima, Peru, October 2020.
Yipeng Zhang, Haofu Liao, Jin Xiao, Nisreen Al Jallad, Jiebo Luo, "A Smartphone-based System for Real-time Early Childhood Caries Diagnosis," Workshop on Perinatal, Preterm and Paediatric Image Analysis, International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), October 2020.
Ray Dorsey, Larsson Omberg, Emma Waddell, Jamie L. Adams, Roy Adams, Mohammad Rafayet Ali, Katherine Amodeo, Abigail Arky, Erika F. Augustine, Karthik Dinesh, Mohammed Ehsan Hoque, Alistair M. Glidden, Stella Jensen-Roberts, Zachary Kabelac, Dina Katabi, Karl Kieburtz, Daniel R. Kinel, Max A. Little, Karlo J. Lizarraga, Taylor Myers, Sara Riggare, Spencer Z. Rosero, Suchi Saria, Giovanni Schifitto, Ruth B. Schneider, Gaurav Sharma, Ira Shoulson, E. Anna Stevenson, Christopher G. Tarolli, Jiebo Luo, Michael P. McDermott, "Deep Phenotyping of Parkinson's Disease," Journal of Parkinson's Disease, in press.
Long Chen, Xinyi Lu, Jianbo Yuan, Joyce Luo, Jiebo Luo, Zidian Xie, and Dongmei Li, "A Social Media Study on Associations of Flavored E-cigarette with Health Symptoms: Observational study," Journal of Medical Internet Research, Vol 22, No 6, 2020.
Haofu Liao, Wei-An Lin, S. Kevin Zhou, Jiebo Luo, "ADN: Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction," IEEE Transactions on Medical Imaging (TMI), Volume 39, Issue 3, 2020.
Jianbo Yuan, Zhiwei Jin, Han Guo, Hongxia Jin, Xianchao Zhang, Tristram Smith, and Jiebo Luo. "Constructing biomedical domain-specific knowledge graph with minimum supervision." Knowledge and Information Systems 62(1): 1-20, 2020.

2019

Jianbo Yuan, Haofu Liao, Rui Luo, Jiebo Luo, "Automated Radiology Report Generation via Multi-view Image Fusion and Medical Concept Enrichment," International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Shenzhen, China, October 2019.
Haofu Liao, Wei-An Lin, Zhimin Huo, William Sehnert, Levon Vogelsang, Kevin Zhou, Jiebo Luo, "Generative Mask Pyramid Network for CT/CBCT Metal Artifact Reduction with Joint Projection-Sinogram Correction," International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Shenzhen, China, October 2019.
Haofu Liao, Wei-An Lin, Jianbo Yuan, S. Kevin Zhou, Jiebo Luo, "Artifact Disentanglement Network for Unsupervised Metal Artifact Reduction," International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Shenzhen, China, October 2019. (Young Scientist Honorable Mention) [PDF][Project Page]
Weijian Li, Viet-Duy Nguyen, Haofu Liao, Matthew Wilder, Ke Cheng, Jiebo Luo, "Patch Transformer for Multi-tagging Whole Slide Histopathology Images," International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), Shenzhen, China, October 2019.
Sen Zhang, Changzheng Zhang, Lanjun Wang, Cixing Li, Dandan Tu, Rui Luo, Guo-Jun Qi, Jiebo Luo, "MSAFusionNet: Multiple Subspace Attention Based Deep Multi-modal Fusion Network," Springer LNCS Proceedings of the 10th International Workshop on Machine Learning in Medical Imaging (MLMI), in conjunction with MICCAI, Shenzhen, China, October 2019.
Dong Liu, Changzheng Zhang, Lanjun Wang, Yaoxin Li, Xiaoshi Chen, Rui Luo, Shuanlong Che, Hehua Liang, Yinghua Li, Si Liu, Dandan Tu, Guo-Jun Qi, Pifu Luo, Jiebo Luo, "DCCL: A Benchmark for Cervical Cytology Analysis," Springer LNCS Proceedings of the 10th International Workshop on Machine Learning in Medical Imaging (MLMI), in conjunction with MICCAI, Shenzhen, China, October 2019.
Zidian Xie, Olga Nikolayeva, Dongmei Li, Jiebo Luo, "Building Risk Prediction Models for Type 2 Diabetes Using Machine Learning Techniques," Preventing Chronic Disease, in press.
Wei-An Lin*, Haofu Liao*, Cheng Peng, Xiaohang Sun, Jingdan Zhang, Jiebo Luo, Rama Chellappa, S. Kevin Zhou, "DuDoNet: Dual Domain Network for CT Metal Artifact Reduction," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019.
Haofu Liao, Wei-An Lin, Jiarui Zhang, Jingdan Zhang, Jiebo Luo, S. Kevin Zhou, "Multiview 2D/3D Rigid Registration via a Point-Of-Interest Network for Tracking and Triangulation," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, June 2019.

2018

S Weisenthal, C Quill, J Luo, H Kautz, S Farooq, M Zand, "A machine learning pipeline to predict acute kidney injury (AKI) in patients without AKI in their most recent hospitalization," Journal of Clinical and Translational Science, 2018.
Haofu Liao, Zhimin Huo, James Sehnert, Kevin S, Zhou, Jiebo Luo, "Adversarial Sparse-View CBCT Artifact Reduction," International Conference on Medical Image Computing and Computer Assisted Intervention (MICCA), Granada, Spain, September 2018. (oral presentation)
Haofu Liao, Yucheng Tang, Gareth Funka-Lea, Jiebo Luo, Kevin S. Zhou, "More Knowledge is Better: Cross-Modality Volume Completion and 3D+2D Segmentation for Intracardiac Echocardiography Contouring," International Conference on Medical Image Computing and Computer Assisted Intervention (MICCA), Granada, Spain, September 2018.
Haofu Liao, Addisu Mesfin, Jiebo Luo, "Joint Vertebrae Identification and Localization in Spinal CT Images by Combining Short- and Long-Range Contextual Information," IEEE Transactions on Medical Imaging 37(5): 1266-1275, 2018.

2017

Xuefeng Peng, Jiebo Luo, Catherine Glenn, Li-Kai Chi, and Jingyao Zhan, "Sleep-deprived Fatigue Pattern Analysis using Large-Scale Selfies from Social Media," Special Session on Intelligent Data Mining, IEEE Big Data Conference, Boston, MA, December 2017.
Yiheng Zhou, Jingyao Zhan and Jiebo Luo, "Predicting Multiple Risky Behaviors via Multimedia Content," International Conference on Social Informatics (SocInfo), Oxford, England, September 2017.
Xitong Yang, Jiebo Luo, "Tracking Illicit Drug Dealing and Abuse on Instagram using Multimodal Analysis," ACM Transactions on Intelligent Systems and Technology, 8(4): 58:1-58:15, August 2017.
Xuefeng Peng and Jiebo Luo, "Large-Scale Sleep Condition Analysis Using Selfies from Social Media," International Conference on Social Computing, Behavioral-Cultural Modeling & Prediction and Behavior Representation in Modeling and Simulation (SBP-BRiMS), Washington, DC, July 2017.
Jianbo Yuan, Chester Holtz, Tristram H Smith, Jiebo Luo, "Autism Spectrum Disorder Detection from Semi-Structured and Unstructured Medical Data," EURASIP Journal on Bioinformatics and Systems Biology, 2017:3, February 2017.
Yiheng Zhou, Catherine Glenn, Jiebo Luo, "Understanding and Predicting Multiple Risky Behaviors from Social Media," AAAI 2017 Joint Workshop on Health Intelligence, San Francisco, CA, February 2017.
Haofu Liao, Jiebo Luo, "A Deep Multitask Learning Approach to Skin Lesion Classification," AAAI 2017 Joint Workshop on Health Intelligence, San Francisco, CA, February 2017.

2016

Haofu Liao, Yuncheng Li, Jiebo Luo, "Skin Disease Classification versus Skin Lesion Characterization: Achieving Robust Diagnosis using Multi-label Deep Neural Networks", International Conference on Pattern Recognition (ICPR), Cancun, Mexico, December 2016.
Kuan Wang, Jiebo Luo, "Detecting Visually Observable Disease Symptoms from Faces", 1ST International Workshop on Biomedical Informatic with Optimization and Machine Learning (BOOM), in conjunction with IJCAI 2016, New York City, New York, July 2016. (Best Paper Runner-Up)
Chunlan Huang, Vincent P. Ness, Xiaochuan Yang, Hongli Chen, Jiebo Luo, Edward B Brown and Xinping Zhang, "Spatiotemporal Analyses of Osteogenesis and Angiogenesis via Intravital Imaging in Cranial Bone Defect Repair," Journal of Bone and Mineral Research, available in PubMed.

2015

Dawei Zhou, Jiebo Luo, Vincent Silenzio, Yun Zhou, Glenn Currier, and Henry Kautz, "Tackling Mental Health by Integrating Unobtrusive Multimodal sensing," the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), Austin, TX, January 25-30, 2015. [PDF]
Junhuan Zhu, Jiebo Luo, Yousuf Khalifar, "Computerized Grading of Cataract Surgery from Videos," Machine Vision and Applications, 26(1): 115-125, 2015.
Yang Cong, Ji Liu, Jiebo Luo, "Deep Sparse Feature Selection for Computer Aided Endoscopy Diagnosis," Pattern Recognition, 48(3): 907-917, 2015.

Pre-2015

Tianli Yu, Jiebo Luo, Narendra Ahuja, "Search strategies for shape regularized active contour," Computer Vision and Image Understanding, 113(10): 1053-1063, 2009.
Hui Luo, Jiebo Luo, "Robust online orientation correction for radiographs in PACS environments," IEEE Transactions on Medical Imaging, 25(10): 1370-1379, 2006.

human computer interaction

Xiaofei Zhou, Hanjia Lyu, Jiebo Luo, Zhen Bai, "ML-SD Modeling: How Machine Learning Can Support Scientific Discovery Learning for K-12 STEM Education," Artificial Intelligence for Education Workshop (AI4Edu), AAAI Conference on Artificial Intelligence (AAAI), February 2023.
Tianran Hu, Anbang Xu, Jiebo Luo. "Touch Your Heart: A Tone-aware Chatbot for Customer Care on Social Media," The ACM CHI Conference on Human Factors in Computing Systems (CHI), Montreal, Canada, April 2018.
Quanzeng You, Jianbo Yuan, Jiaqi Wang, Philip Guo, Jiebo Luo. Snap n' Shop: Visual Search-Based Mobile Shopping Made a Breeze by Machine and Crowd Intelligence, IEEE International Conference on Semantic Computing (ICSC), February 2015. [PDF]
Vivek K. Singh, Jiebo Luo, Dhiraj Joshi, Phoury Lei, Madirakshi Das, Peter Stubler, "Reliving on demand: a total viewer experience," ACM international conference on Multimedia, November 2011.
Dhruv Batra, Adarsh Kowdle, Devi Parikh, Jiebo Luo, and Tsuhan Chen, "iCoSeg: Interactive Co-segmentation of Objects in Image Collections," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, June 2010.

mobile and pervasive computing

Haidong Zhao, Lina Zhang, M. B. Kirkham, Stephen M. Welch, John W. Nielsen-Gammon, Guihua Bai, Jiebo Luo, Daniel A. Andresen, Charles W. Rice, Nenghan Wan, Romulo P. Lollato, Dianfeng Zheng, Prasanna H. Gowda, Xiaomao Lin, "U.S. Winter Wheat Yield Loss Attributed to Compound Hot-dry-windy Events," Nature Communications, 13, Article number: 7233, 2022.
Yipeng Zhang, Haofu Liao, Jin Xiao, Nisreen Al Jallad, Jiebo Luo, "A Smartphone-based System for Real-time Early Childhood Caries Diagnosis," Workshop on Perinatal, Preterm and Paediatric Image Analysis, International Conference on Medical Image Computing and Computer Assisted Interventions (MICCAI), October 2020.
Young Chol Song, Henry Kautz, James Allen, Mary Swift, Yuncheng Li, Jiebo Luo, "A Markov Logic Framework for Recognizing Complex Events from Multimodal Data," ACM International Conference on Multimodal Interaction, Sidney, Australia, December 2013.
Heng Liu, Tao Mei, Jiebo Luo, Houqiang Li, Shipeng Li, "Finding Perfect Rendezvous On the Go: Accurate Mobile Visual Localization and Its Applications to Routing," ACM Multimedia Conference, Nara, Japan, October 2012. (Long paper, Best Paper Candidate)

Pixels and Patches (Bits and Bytes)

Pictures (Featured Big Picture Talks)

Multimodal LLMs as Social Media Analysis Engines
Recent research has offered insights into the extraordinary capabilities of Multimodal Large Multimodal Models (MLMMs) in various general vision and language tasks. There is growing interest in how MLMMs perform in more specialized domains. Social media content, inherently multimodal, blends text, images, videos, and sometimes audio. To effectively understand such content, models need to interpret the intricate interactions between these diverse communication modalities and their impact on the conveyed message. Understanding social multimedia content remains a challenging problem for contemporary machine learning frameworks. To evaluate MLLMs' capabilities for social multimedia analysis, we select five representative tasks, including sentiment analysis, hate speech detection, fake news identification, demographic inference, and political ideology detection. Our investigation begins with a preliminary quantitative analysis for each task using existing benchmark datasets, followed by a careful review of the results and a selection of qualitative samples that illustrate GPT-4V's potential in understanding multimodal social media content. GPT-4V demonstrates remarkable efficacy in these tasks, showcasing strengths such as joint understanding of image-text pairs, contextual and cultural awareness, and extensive commonsense knowledge. In addition to the known hallucination problem, notable challenges remain as GPT-4V struggles with tasks involving multilingual social multimedia comprehension and has difficulties in generalizing to the latest trends in social media. We further present several attempts to improve the performance on some tasks. The insights gleaned from our findings underscore a promising future for MLMMs in enhancing our understanding of social media content and its users through the analysis of multimodal information.

Temporal Learning for Video-Language Understanding and Generation
Research in video-language understanding and generation brings together the subfields of computer vision (CV) and natural language processing (NLP) in AI. Significant progress has been made in image-language understanding and generation studies. However, progress in the video-text domain lags behind. The main challenge posed by video is how to model temporal dynamics. We address this challenge in three directions: (1) learning video-language alignment, exploring both video and language aspects, (2) learning grammar induction with video, leveraging videos for language aquisition, and (3) learning video generation with text, producing good quality results efficiently.

What ChatGPT Can and Cannot Do, and More on AIGC
In late 2022, OpenAl launched ChatGPT. In addition to being able to write and chat coherently, ChatGPT has demonstrated the ability to answer questions in multiple domains, marking a major step forward for generative Al (AIGC). Many industries, such as technology, media, law, healthcare, education, and finance, are embracing the challenges and opportunities of ChatGPT and AIGC. From a research point of view, what tasks ChatGPT is capable of, what capabilities it lacks, and how generative Al will impact human life and work, are pressing topics worth investigating.

Vision and Language: Past, Present, and Future
Computer vision and natural language processing are two key branches of artificial intelligence. Since the goal of computer vision has always been automatic extraction, analysis, and understanding of useful information from a single image or a sequence of images, it is natural for vision and language to come together to enable high-level computer vision tasks. Conversely, information extracted from images and videos can facilitate natural language processing tasks. Recent advances in machine learning and deep learning are facilitating reasoning about images and text in a joint fashion. in this talk, we will review a recently active area of research at the intersection of vision and language, including video-language alignment, image and video captioning, visual question answering, image retrieval using complex text queries, image generation from textual descriptions, language grounding in images and videos, as well as multimodal machine translation and vision-aided grammar induction.

COVID-19: What Social Media and Machine Learning Can Inform Us
The COVID-19 pandemic has severely affected people's daily lives and caused tremendous economic losses worldwide. However, its influence on public opinions and people's mental health conditions has not received as much attention. In addition, the related literature in these fields has primarily relied on interviews or surveys, largely limited to small-scale observations. In contrast, the rise of social media provides an opportunity to study many aspects of a pandemic at scale and in real-time. Meanwhile, the recent advances in machine learning and data mining allow us to perform automated data processing and analysis. We will introduce several recent studies ranging from 1) characterizing Twitter users and topics regarding the use of controversial terms for COVID-19, 2) understanding how college students respond differently than the general public to the pandemic, 3) monitoring depression trends throughout COVID-19, to 4) studying consumer hoarding behaviors during the pandemic.

Learning with Unpaired Data
Many learning tasks can be summarized as learning a mapping from a structured input to a structured output, such as machine translation, image style transfer, image captioning, and image dehazing. Such mappings are usually learned on paired training data, where an input sample and its corresponding output are both provided. Collecting paired training data often involves expensive human annotation, and the scale of paired training data is therefore often limited. As a result, the generalization ability of models trained on paired data is also limited. One way to mitigate this issue is learning with unpaired data, which is far less expensive to collect. Taking machine translation as an example, the unpaired training data can be collected separately from newspapers in the source language and target language without any annotation. The challenge of unpaired learning turns into how to align the unpaired data. With carefully designed objectives, unpaired learning has achieved remarkable progress on several tasks. This talk will cover the data collection and training methods of several unpaired learning tasks to illustrate the power of learning with unpaired data.

Spear and Shield: War on Misinformation
Social media, including Twitter and Chinese Weibo, has become an important channel of news and public opinion in recent years. For example, during the 2016 U.S. presidential election, candidates and their supporters were actively engaged on Twitter to run campaigns and express their opinions. The convenience and openness of social media have also fostered various misinformation, rumors, and fake news, which have become a serious public concern. To increase the credibility of information on social media and prevent the spreading of fake contents, it is crucial to detect misinformation and disinformation automatically. We introduce our work in this area and provide analyses of the impacts of misinformation on major events such as the U.S. presidential election and COVID-19 pandemic.

Jointly Understanding User and Content from Multimodal Social Media Data
Social media has become prevalent in our lives as the main venue for sharing information, recording moments, as well as expressing feelings and creativity. Two of the most important signals in social media data are user profiles and user sentiments, both of which can be inferred from multimodal social media content that users choose to share or respond to. We have been employing a joint modeling and understanding approach to building interpretable user profiles and characterizing nuanced user sentiments for a wide variety of applications including Instagram, Tiktok, Facebook, Pinterest, Youtube, Twitter, and Wechat. In this talk, we will present the framework along with a few case studies to illustrate the power of our approach, as well as its utilities in content creation, promotion, and recommendation.

Computer Vision ++: The Next Step towards Big AI
With the huge successes of deep learning in computer vision, many vision problems are seemingly being solved. Where do we go from here? We will discuss a few directions where computer vision can be either further pushed to deal with data scarcity and data noise, or synergistically integrated with other disciplines such as NLP and data mining, to continue to advance the frontiers of artificial intelligence.

Harvesting the Healing Power of AI and Big Data
With the recent rapid advances in artificial intelligence, machine learning and data science, a promising domain is healthcare and wellness management. In this talk, we discuss two directions: first, leveraging electronic medical data including unstructured forms and videos to build robust algorithms for disease diagnosis and medical training, and second and more interestingly, utilizing social media data to detect and alter user unhealthy behaviors such as drinking, smoking, drug-use, and eating disorder. It is also intriguing how these two directions can be combined synergistically, and further to implement effective and innovative intervention.

When Computer Vision meets E-Commerce
Two of the top 10 largest companies by market cap in the world, namely Amazon and Alibaba, are e-commerce companies that are enjoying huge success. At the same time, computer vision and artificial intelligence are making strides in both technologies and applications. What power can be unleashed when computer vision meets e-commerce. We will present a few recent advances in user profiling, behavior analytics, product recommendation, as well as deep sales data mining.

Video and Language
Video has become ubiquitous on the Internet, TV, as well as personal devices. Recognition of video content has been a fundamental challenge in computer vision for decades, where previous research predominantly focused on recognizing videos using a predefined yet limited vocabulary. Thanks to the recent development of deep learning techniques, researchers in multiple communities are now striving to bridge videos with natural language in order to move beyond classification to interpretation, which should be regarded as the ultimate goal of video understanding. We will present recent advances in exploring the synergy of video understanding and language processing techniques, including video-language alignment, video captioning, and video emotion analysis.

Computational Inference of Emotion in Images
With the recent successes in using deep learning techniques to solve computer vision problems, the performances of the state of the art algorithms in many areas, especially object recognition, have been dramatically improved. Researchers are now inspired to address yet more challenging problems, such as associating pictures with aesthetics, and have also reported progress. One remaining final frontier in extracting meaning from images is related to the recognition of emotions that images arouse in humans. The key challenges are the loose and highly abstract nature of semantics associated with emotions. We will discuss how to effectively employ a data-intensive approach to emotion recognition in images, as well as multimedia that include both image and text information.

You Are What You Post: Personal Analytics from Big Social Media Data
From the abundance of personal records, public data, and user-generated social multimedia contents, rich information can be inferred regarding user profile, user interests, user sentiment, user behaviors, user mobility, user lifestyles, as well as user physical and mental health. We will present a few recent advances in this important arena.

2016 Presidential Election: Donald Trump De-mystified
With social media being increasingly utilized to boost political campaigns, Donald Trump looms large in the 2016 US Presidential Election. With the ability to track the Twitter followers of the major candidates in the running, we conduct a series of in-depth, fine-grained, data-driven analyses, including 1) Will Sanders Supporters Jump Ship for Trump? Fine-grained Analysis of Twitter Followers?, 2) Pricing the Woman Card: Gender Politics between Hillary Clinton and Donald Trump, and 3) Voting with Feet: Who are Leaving Hillary Clinton and Donald Trump? These studies represent a new paradigm for studying political campaigns and provide valuable insight at unprecedented scales and in real-time.

Big Data Better Life
We live in the age of big data. The biggest big data is big visual data, which includes images and other associated information. The biggest challenge is to develop effective computational methods for making sense of such massive visual data. Unlike text which is clean, segmented, compact, one dimensional and indexable, visual content is noisy, unsegmented, high entropy and multidimensional. In this talk, we present a few recent advances towards the ultimate goal of using big data, in particular large-scale and rich multi-modality data, to achieve robust intelligence in order to understand and improve life in terms of healthcare, well-being, politics, business, infotainment, and so on.

Vision with A Billion Eyes
A recent trend in computer vision is driven by images and video generated by heterogeneous and multi-perspective visual sensing networks. We present a few examples of research along this line. First, we will present an interesting framework for event recognition. With GPS information, we obtain satellite images corresponding to picture locations and investigate their novel use to recognize the picture-taking environment. We then combine this inference with classical vision-based event detection methods and demonstrate the synergistic fusion of the two approaches. Second, to determine the viewing direction for geotagged photos, we utilize both Google StreetView and Google Earth satellite images. Third, we explore using phone-captured images for localization as it contains more context information than the embedded sensory GPS coordinates. We then build applications to enable people to enjoy ubiquitous location-based services (LBS) using their phones. Fourth, we leverage crowd-sourced photos to remove unwanted bystanders from tourist photos taken at popular attractions and measure air pollution in major cities in China. Furthermore, given a new source of visual data from public webcams deployed in urban environments, we will present some ongoing work on crowd analytics using such data.

Social Multimedia as Sensors
Social multimedia can be exploited as a powerful new way of sensing social behaviors and activities from user-generated social multimedia contents, including building user profiles from a user's personal photo collection, inferring user personality traits from social media language usage, producing popular and diverse tourism routes from crowd-sourced geo-tagged photos, extracting user sentiment from both textual and visual information in social media, monitoring risky behaviors such as teenage drinking and drug abuse, and forecasting election outcome based on image sharing activities and user demographics extracted using computer vision techniques. Finally, we will share thoughts on current challenges and future directions.
Using Social Media to Manage Mental Health
"Computer Vision with a Third Eye" (Luo)
"Data Mining in Social Multimedia Networks" (starting at 1 '32 "33) (Luo)
"Thinking in Network Terms" (Barabasi)
"HCI-orientated Social Computing" (Wang)

Resources

Research Themes [for example projects, check my PhD students' homepages]

Computer vision: recognition of objects, scenes, people, locations, actions and events from images and video
Vision and language: description and explanation of visual content; language-based search and retrieval
Social media data mining: prediction, forecasting, profiling, and recommendation using open source data
Machine learning: learning with large scale loosely labeled web data, cross-domain learning, few-shot learning
Human computer interaction: interactive visual recognition; integration of crowd and machine intelligence
Biomedical informatics: healthcare and wellness analytics using text and visual data; surgical video analysis
Mobile / Pervasive computing: context-aware applications; multimodal inference from multiple sensors
Computational photography: computer vision-based 2D/3D enhancement; augmented/mobile photography
Digital multimedia experience: multimodal reliving; aesthetics, emotion, sentiment, and influence of multimedia

VIStA Research Group Members:

Visiting Researchers

Prof. Jinsong Su
(Xiamen University)

PhD Students

Yuncheng Li

Quanzeng You

(Snapchat Research 2017-2019) (Microsoft Research AI, 2017-2023)
(Google Brain, 2019- ) (Tiktok, 2023- )

Tianran Hu

Jianbo Yuan

(College of William & Mary) (Amazon - AWS, 2019-2021)
(Tiktok, 2021- )

Yu Wang (PSC)

Yang Feng

Haofu Liao

(Apple Siri, Pinterest) (Goolge Cloud) (Amazon - AWS)

Tianlang Chen

Zhengyuan Yang

(Amazon - AWS) (Microsoft Research)

Wei Xiong

Weijian Li

(Adobe Photoshop) (Amazon Go)

Songyang Zhang

Songyang Zhang

Wei Zhu

Haitian Zheng

(Amazon Research)       (Amazon Research)        (Adobe Research)

Jie An Hang Hua

Hanjia Lyu

Jingyang Lin Yuan Yao

Daoan Zhang Jinfa Huang Yongsheng Yu

   Zhenghong Zhou

Visiting PhD Students



      Zixuan Wang

MS Students

CS Program: Varad Thikekar, Ziyun Zeng, Satvik Garg

Data Science Program: Krishna Saimahith Yemmanuru, Yiyao Tao, Hengyu Zhang, Shaleen Ishfar

Undergraduate Students

CS/DS Program: Junhua Huang, Yi Wu, Yuqing Fan, Yumo Fan, Wanting Shan, Hanzhang Yin, Jiarui Wu, Michael Reidy
NSF REU program: Ann Nanyunja (UMass), Christiana Murph (UTK), A'nyar Car (Kean University), Destiny Medearis (San Francisco State University), Zaria Kangethe (Moraine Valley Community College), Olivia Lopez (Univ. of South Carolina), Michael Zhou (Cornell), Miles Sigel (Rice)

Alumni

Visiting Professors

Prof. Yang Cong (SIA)
Prof. Tian-Liang Liu (NUPT)
Prof. Jun Wu (BJYU)
Prof. Congyan Lang (BJTU)
Prof. Weining Wang (SCUT)
Prof. Yi Wang (DLUT)

Visiting Postdocs

Dr. Kuan-Ting Chen (NTU)

Visiting PhD Students

Kuan-Ting Chen (NTU)
Wu Liu (ICT, Chinese Academy of Sciences)
Zhiwei Jin (ICT, Chinese Academy of Sciences)
Xiangyang Xu (Nanjing University)
Jesse Wang (UR Medicine)
Luigi Celona (Universita degli Studi di Milano-Bicocca)
Mengshi Qi (Beihang University)
Wenbin Li (Nanjing University)
Jianxin Lin (USTC)
Jing Wang (NJUST)
Mingkun Yang (HUST)

Former Graduate Students

CS PhD Program: Lam Tran, Spyridoula Gravani (PhD@Rochester), Junhuan Zhu (LinkedIn), Qiguang Liu (Youtube), Jie Chen (Microsoft), Zhaoyi Wan, Yuqi Gao
DS MS Program: Weihong Qi, Qinqin Xiao, Edinam Klutse, Samuel Nuamah-Amoabeng, Junyu Chen, Matthew Johnston, Mohamad Ali Kalassina, Hanjia Lyu, Ravi Dugh, Yangxin Fan, Tangqiu Jiang, Ziyu Xiong, Meghana Murthy, Trang Nguyen, Xupin Zhang, Jong Hwi Park, Kefei Wu
CS MS Program: Jinsheng Pan, Zichen Wang, Arjun Krishnan, Jayant Kumar, Arsal Imtiaz, Ashwin Ramesh, Henry Moncure, Haozheng Du, Junda Wang, Viet Duong, Sai Anirudh Reddy Avilala, Rohit Venkata Sai Dulam, Zhezheng Hong, Yubao Liu, Yuxiao Chen (PhD@Rutgers), Tianxin Xie*, Kshitij Joshi*, Yiyun Peng*, Hao Zhou, Lele Chen, Rijul Magu, He Gong, Eric Biglow, Wentao Cai (PhD@Rochester), Xitong Yang (PhD@UMD), Haoyuan Xiao, Ran Pang (Bloomberg), Yunping Shao, Bella Wang, Han Guo (ICT)
ECE MS Program: Beibei Qi, Yixuan Zhang, Zhongping Zhang, Taolue Chen, Yuchen Wu, Peizhong Cong, Siddarth Patki, Zhiyuan Li, Lin Wang, Jile Hu, Liangkai Zhang, Jiong Liu (Epic), Agnes Zhang, Jingwei Guo, Jiaqi Wang, Yao Zhou (PhD@ASU), Dawei Zhou (PhD@ASU), Yun Zhou, Ge Ma (PhD@Brandeis), Yingru Tian
TEAM Program: Qi Tang, Shujing Yu, Augustine Baretto (start-up), Bowei Zhang

Former Undergraduate Students

CS/DS Program: Alexander Martin, Yiwei Han, Jixuan Leng, Junfei Liu, Jiayi Hao, Yulin Feng, Runzhou Liu, Yufei Zhao, Miles Zhou, Quan Luu, Grace Julien, Nikola Danevski, Enting Zhou, Yurong Liu, Jingyuan Chen, Emile Hemway, Peirong Hao, Zachary Mustin, Brian Kimani, Sidhant Bendre, Anisha Bhattacharya, Alexandra Ferrante, Zihe Zheng, Neil Yeung, Jonathan Lai, Wei Wu, Jinglong Ruan, Bokai Zhang, Li Sun, Haoqi Zhang, Dawei Zhang, Tongyu Yang, Edan Meyer, Yipeng Zhang, Long Chen, Xinyi Lu, Shuaidong Pan, Faner Lin, Zhifan Nan, Gao Fan, Tolga Atkas, Viet Duy Nuygen, Viet Duong, Phu Pham, Yutong He, Benjamin King, Peijun Xu, Yiming Pan, David Anuta, Josh Churchin, Numair Sani, Ryan Berger, Kelly He (Ernst & Young), Trevor Whitestone, Chester Holtz (UCSD), Yiheng Zhou (CMU), Yuntao Zhou (CMU), Honglin Zheng (UCLA), Xuefeng Peng (Harvard), Kevin Chi, Zhishen Pan, Jiagen Zheng (UCSD), Jingyao Zhan (UCSD), Zoe Tiet, Raina Langevin, Jake Schechner, Caesar De Hoyos, Kuan Wang (PhD@U Penn), Lee Murphy, Jacob Niebloom, Kezhen Chen (PhD@Northwestern), Chunpai Wang (PhD@Albany), Andrew Nocka (Charles Schwab), Simon Weber (Hacker School)
Math/Stats/Applied Math Program: Danning Zheng (Bloomberg)
Economy Program: Alex Feiszli

Prospective PhD Students

I'm always looking for self-motivated students and usually recruit two new PhD students per year. Applicants are expected to have strong skills in both programming and mathematics, as well as research experience in closely related areas. Grades alone won't cut it. "I worked on a gazillion of projects" (but have no evidence) is a no-no. Research publications in ranked venues of computer vision, machine learning, or data mining is a plus (and the only reason you need to contact me directly). Sorry but generic spams will be ignored.

To apply, check http://www.cs.rochester.edu/graduate/phd-program.

Visiting Students

I normally have two to three self-funded visiting student positions per year. I do not provide any paid visiting or postdoc positions, and I do not consider any visits shorter than 3 months. It is expected that you have some minimum prior research experience. A recommendation letter via email directly from your research advisor is needed, if you have not personally met me.

Current UR Students

Current UR students (undergraduate or MS) interested in working with me are welcome to stop by my office during my office hours (see the posting outside my office). It is desirable that you have taken a course with me prior to starting any research in my lab. A resume through email in advance is recommended as opposed to a cold knock on the door.

Reference Letter Seekers

You must have your advisor/mentor vouch for you, if you have not personally worked with me. If you intend to work in my lab, producing meaningful or publishable results is the essence for the reference letter as a generic letter stating that you
worked in my lab would not help your graduate school application.

Internship Seekers

In general, we do not host summer interns unless there is a preexisting arrangement, or you have prior research product(s) to validate your seriousness.

Why URCS?

The University of Rochester is a private institution that was founded in 1850. It is one of the smallest and most collegiate schools among the nation's top research universities. According to US News and World Report 2019, the University of Rochester's ranking in the Best National Universities is 29th. It is also one of only 25 schools named a "New Ivy" in the 2007 Kaplan/Newsweek "How to Get into College Guide".

The Department of Computer Science at the University of Rochester is a research-oriented department with a distinguished history of contributions in systems, theory, artificial intelligence, and HCI. Historically, a third of its PhD graduates have received tenure-track faculty positions, and its alumni include prominent leaders at major research laboratories such as Google, Microsoft, and IBM. Over the past 50 years, URCS has built a strong reputation in AI as evidenced by the accomplishments, awards, and accolades in AI by the URCS faculty members.

Research is a Journey not for the Faint-hearted

Do you have what it takes to be a successful PhD student? The initial threshold is 4 out of 7 and I hope you have all 7 at graduation.

Grit 坚毅
Zest 激情
Self-control 自制力
Optimism 乐观态度
Gratitude 感恩精神
Social intelligence 社交智力
Curiosity 好奇心

Curious how I evalaute students?

Be the best you can be. You can be the best.

Battle Hymns

Hall of Fame (The Script)

Exchange

Prof. Eckehard Steinbach (08/29/2013)

Prof. Honglak Lee (10/14/2013)

Prof. Ruslan Salakhutdinov (10/17/2013)

Prof. Bill Freeman (10/25/2013)

Prof. James Rehg (10/02/2014)

Free access and statistics provided by the ACM Digital Library

Back to the top

News

Biography

Teaching

Major Professional Services

Books

Selected Recent Publications (Publications listed more comprehensively by DBLP)

My Citations

surveys

Aesthetics and Emotions in Images: A Computational Perspective

Geotagging in Multimedia and Computer Vision -- A Survey

Small Data Challenges in Big Data Era: Recent Progress on Unsupervised and Semi-Supervised Methods

Human Behavior in the Time of COVID-19: Learning from Big Data

computer vision, NLP and media understanding and generation

machine learning

web and social media data mining

biomedical analytics and health informatics

human computer interaction

mobile and pervasive computing

Pictures (Featured Big Picture Talks)

Resources

Research Themes [for example projects, check my PhD students' homepages]

Exchange